2026-04-10 18:13:37,540 [ 57768 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse (runner:53, check_args_and_update_paths) 2026-04-10 18:13:37,540 [ 57768 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:79, check_args_and_update_paths) 2026-04-10 18:13:37,540 [ 57768 ] INFO : utils dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/utils (runner:90, check_args_and_update_paths) 2026-04-10 18:13:37,540 [ 57768 ] INFO : base_configs_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:92, check_args_and_update_paths) clickhouse_integration_tests_volume Running pytest container as: 'docker run --rm --name clickhouse_integration_tests_09sh6d --privileged --dns-search='.' --memory=30709030912 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=5dc43a6382f0 -e DOCKER_BASE_TAG=5ccda723c1fc -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=d862517635bf -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e CLICKHOUSE_USE_OLD_ANALYZER=1 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 --report-log=parallel0_0.jsonl --report-log-exclude-logs-on-passed-tests test_accept_invalid_certificate/test.py::test_accept test_accept_invalid_certificate/test.py::test_connection_accept test_accept_invalid_certificate/test.py::test_default test_accept_invalid_certificate/test.py::test_strict_connection_reject test_accept_invalid_certificate/test.py::test_strict_reject test_accept_invalid_certificate/test.py::test_strict_reject_with_config test_asynchronous_metric_log_table/test.py::test_event_time_microseconds_field test_attach_partition_using_copy/test.py::test_all_replicated test_attach_partition_using_copy/test.py::test_both_mergetree test_attach_partition_using_copy/test.py::test_not_work_on_different_disk test_attach_partition_using_copy/test.py::test_only_destination_replicated test_backup_restore_azure_blob_storage/test.py::test_backup_restore test_backup_restore_azure_blob_storage/test.py::test_backup_restore_correct_block_ids test_backup_restore_azure_blob_storage/test.py::test_backup_restore_diff_container test_backup_restore_azure_blob_storage/test.py::test_backup_restore_on_merge_tree test_backup_restore_azure_blob_storage/test.py::test_backup_restore_with_named_collection_azure_conf1 test_backup_restore_azure_blob_storage/test.py::test_backup_restore_with_named_collection_azure_conf2 'test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-default]' 'test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-s3_native_copy]' 'test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-s3_no_native_copy]' 'test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-default]' 'test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-s3_native_copy]' 'test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-s3_no_native_copy]' test_backward_compatibility/test_ip_types_binary_compatibility.py::test_ip_types_binary_compatibility test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability test_backward_compatibility/test_short_strings_aggregation.py::test_backward_compatability test_buffer_profile/test.py::test_buffer_profile test_buffer_profile/test.py::test_default_profile test_check_table_name_length/test.py::test_backward_compatibility test_check_table_name_length/test.py::test_check_table_name_length test_cleanup_after_start/test.py::test_old_dirs_cleanup test_cluster_discovery/test_auxiliary_keeper.py::test_cluster_discovery_with_auxiliary_keeper_startup_and_stop test_cluster_discovery/test_password.py::test_connect_with_password test_compatibility_merge_tree_settings/test.py::test_check_projections_compatibility test_compatibility_merge_tree_settings/test.py::test_config_overrides_compatibility test_compression_nested_columns/test.py::test_nested_compression_codec test_concurrent_queries_for_user_restriction/test.py::test_exception_message test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool test_config_reloader_interval/test.py::test_reload_config test_config_xml_main/test.py::test_xml_main_conf test_config_yaml_merge_keys/test.py::test_yaml_merge_keys_conf 'test_cow_policy/test.py::test_cow_policy[cow_policy_multi_disk]' 'test_cow_policy/test.py::test_cow_policy[cow_policy_multi_volume]' test_custom_settings/test.py::test_custom_settings test_custom_settings/test.py::test_illformed_setting test_ddl_alter_query/test.py::test_alter test_ddl_alter_query/test.py::test_ddl_queue_hostname_change test_ddl_worker_replicas/test.py::test_ddl_worker_replicas test_default_role/test.py::test_alter_user test_default_role/test.py::test_set_default_roles test_default_role/test.py::test_wrong_set_default_role 'test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_complex[complex_key_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_ranged[range_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_simple[hashed]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_cache-False-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_cache-True-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_direct-False-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_direct-True-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_hashed-False-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_hashed-True-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_ranged[range_hashed-False-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_ranged[range_hashed-True-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[cache-False-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[cache-True-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[direct-False-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[direct-True-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[flat-False-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[flat-True-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[hashed-False-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[hashed-True-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[cache-False-True]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[cache-True-True]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[direct-False-True]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[direct-True-True]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[flat-False-True]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[flat-True-True]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[hashed-False-True]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[hashed-True-True]' test_distributed_ddl_password/test.py::test_alter test_distributed_ddl_password/test.py::test_truncate test_distributed_directory_monitor_split_batch_on_failure/test.py::test_distributed_background_insert_split_batch_on_failure_OFF test_distributed_directory_monitor_split_batch_on_failure/test.py::test_distributed_background_insert_split_batch_on_failure_ON test_executable_user_defined_function/test.py::test_executable_function_always_error_python test_executable_user_defined_function/test.py::test_executable_function_argument_python test_executable_user_defined_function/test.py::test_executable_function_bash test_executable_user_defined_function/test.py::test_executable_function_input_nullable_python test_executable_user_defined_function/test.py::test_executable_function_non_direct_bash test_executable_user_defined_function/test.py::test_executable_function_parameter_python test_executable_user_defined_function/test.py::test_executable_function_python test_executable_user_defined_function/test.py::test_executable_function_query_cache test_executable_user_defined_function/test.py::test_executable_function_send_chunk_header_python test_executable_user_defined_function/test.py::test_executable_function_signalled_python test_executable_user_defined_function/test.py::test_executable_function_slow_python test_executable_user_defined_function/test.py::test_executable_function_sum_json_python test_executable_user_defined_function/test.py::test_executable_function_sum_python test_file_cluster/test.py::test_count test_file_cluster/test.py::test_format_detection -vvv " altinityinfra/integration-tests-runner:226bfaf75ac1 '. Start tests ============================= test session starts ============================== platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /usr/bin/python3 cachedir: .pytest_cache Test order randomisation NOT enabled. Enable with --random-order or --random-order-bucket= rootdir: /ClickHouse/tests/integration configfile: pytest.ini plugins: timeout-2.3.1, repeat-0.9.3, order-1.0.0, reportlog-0.4.0, xdist-3.5.0, random-order-1.1.1 timeout: 900.0s timeout method: signal timeout func_only: False created: 10/10 workers 10 workers [100 items] scheduling tests via LoadFileScheduling test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_cache-False-False] test_executable_user_defined_function/test.py::test_executable_function_always_error_python test_buffer_profile/test.py::test_buffer_profile test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_complex[complex_key_hashed] test_accept_invalid_certificate/test.py::test_accept test_attach_partition_using_copy/test.py::test_all_replicated test_default_role/test.py::test_alter_user test_backup_restore_azure_blob_storage/test.py::test_backup_restore test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-default] test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool [gw4] [ 1%] PASSED test_accept_invalid_certificate/test.py::test_accept test_accept_invalid_certificate/test.py::test_connection_accept [gw4] [ 2%] PASSED test_accept_invalid_certificate/test.py::test_connection_accept test_accept_invalid_certificate/test.py::test_default [gw4] [ 3%] PASSED test_accept_invalid_certificate/test.py::test_default test_accept_invalid_certificate/test.py::test_strict_connection_reject [gw9] [ 4%] PASSED test_buffer_profile/test.py::test_buffer_profile test_buffer_profile/test.py::test_default_profile [gw4] [ 5%] PASSED test_accept_invalid_certificate/test.py::test_strict_connection_reject test_accept_invalid_certificate/test.py::test_strict_reject [gw4] [ 6%] PASSED test_accept_invalid_certificate/test.py::test_strict_reject test_accept_invalid_certificate/test.py::test_strict_reject_with_config [gw4] [ 7%] PASSED test_accept_invalid_certificate/test.py::test_strict_reject_with_config [gw9] [ 8%] PASSED test_buffer_profile/test.py::test_default_profile [gw7] [ 9%] PASSED test_default_role/test.py::test_alter_user test_default_role/test.py::test_set_default_roles [gw1] [ 10%] PASSED test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-default] test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-s3_native_copy] test_compatibility_merge_tree_settings/test.py::test_check_projections_compatibility test_check_table_name_length/test.py::test_backward_compatibility [gw1] [ 11%] PASSED test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-s3_native_copy] test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-s3_no_native_copy] [gw7] [ 12%] PASSED test_default_role/test.py::test_set_default_roles test_default_role/test.py::test_wrong_set_default_role [gw1] [ 13%] PASSED test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-s3_no_native_copy] test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-default] [gw0] [ 14%] PASSED test_executable_user_defined_function/test.py::test_executable_function_always_error_python test_executable_user_defined_function/test.py::test_executable_function_argument_python [gw7] [ 15%] PASSED test_default_role/test.py::test_wrong_set_default_role [gw1] [ 16%] PASSED test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-default] test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-s3_native_copy] [gw0] [ 17%] PASSED test_executable_user_defined_function/test.py::test_executable_function_argument_python test_executable_user_defined_function/test.py::test_executable_function_bash [gw1] [ 18%] PASSED test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-s3_native_copy] test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-s3_no_native_copy] [gw0] [ 19%] PASSED test_executable_user_defined_function/test.py::test_executable_function_bash test_executable_user_defined_function/test.py::test_executable_function_input_nullable_python test_cow_policy/test.py::test_cow_policy[cow_policy_multi_disk] [gw1] [ 20%] PASSED test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-s3_no_native_copy] [gw0] [ 21%] PASSED test_executable_user_defined_function/test.py::test_executable_function_input_nullable_python test_executable_user_defined_function/test.py::test_executable_function_non_direct_bash [gw0] [ 22%] PASSED test_executable_user_defined_function/test.py::test_executable_function_non_direct_bash test_executable_user_defined_function/test.py::test_executable_function_parameter_python [gw0] [ 23%] PASSED test_executable_user_defined_function/test.py::test_executable_function_parameter_python test_executable_user_defined_function/test.py::test_executable_function_python [gw0] [ 24%] PASSED test_executable_user_defined_function/test.py::test_executable_function_python test_executable_user_defined_function/test.py::test_executable_function_query_cache [gw9] [ 25%] PASSED test_check_table_name_length/test.py::test_backward_compatibility test_check_table_name_length/test.py::test_check_table_name_length [gw0] [ 26%] PASSED test_executable_user_defined_function/test.py::test_executable_function_query_cache test_executable_user_defined_function/test.py::test_executable_function_send_chunk_header_python [gw9] [ 27%] PASSED test_check_table_name_length/test.py::test_check_table_name_length [gw4] [ 28%] PASSED test_compatibility_merge_tree_settings/test.py::test_check_projections_compatibility test_compatibility_merge_tree_settings/test.py::test_config_overrides_compatibility [gw4] [ 29%] PASSED test_compatibility_merge_tree_settings/test.py::test_config_overrides_compatibility [gw0] [ 30%] PASSED test_executable_user_defined_function/test.py::test_executable_function_send_chunk_header_python test_executable_user_defined_function/test.py::test_executable_function_signalled_python [gw0] [ 31%] PASSED test_executable_user_defined_function/test.py::test_executable_function_signalled_python test_executable_user_defined_function/test.py::test_executable_function_slow_python test_ddl_alter_query/test.py::test_alter test_distributed_ddl_password/test.py::test_alter test_custom_settings/test.py::test_custom_settings [gw0] [ 32%] PASSED test_executable_user_defined_function/test.py::test_executable_function_slow_python test_executable_user_defined_function/test.py::test_executable_function_sum_json_python [gw6] [ 33%] PASSED test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated [gw0] [ 34%] PASSED test_executable_user_defined_function/test.py::test_executable_function_sum_json_python test_executable_user_defined_function/test.py::test_executable_function_sum_python [gw8] [ 35%] PASSED test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_complex[complex_key_hashed] test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_ranged[range_hashed] [gw0] [ 36%] PASSED test_executable_user_defined_function/test.py::test_executable_function_sum_python [gw4] [ 37%] PASSED test_custom_settings/test.py::test_custom_settings test_custom_settings/test.py::test_illformed_setting [gw4] [ 38%] PASSED test_custom_settings/test.py::test_illformed_setting test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability test_backward_compatibility/test_ip_types_binary_compatibility.py::test_ip_types_binary_compatibility [gw9] [ 39%] PASSED test_ddl_alter_query/test.py::test_alter test_ddl_alter_query/test.py::test_ddl_queue_hostname_change [gw1] [ 40%] PASSED test_distributed_ddl_password/test.py::test_alter test_distributed_ddl_password/test.py::test_truncate [gw8] [ 41%] PASSED test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_ranged[range_hashed] test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_simple[hashed] [gw9] [ 42%] PASSED test_ddl_alter_query/test.py::test_ddl_queue_hostname_change [gw3] [ 43%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_cache-False-False] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_direct-False-False] [gw1] [ 44%] PASSED test_distributed_ddl_password/test.py::test_truncate test_asynchronous_metric_log_table/test.py::test_event_time_microseconds_field test_file_cluster/test.py::test_count [gw1] [ 45%] PASSED test_asynchronous_metric_log_table/test.py::test_event_time_microseconds_field test_concurrent_queries_for_user_restriction/test.py::test_exception_message [gw9] [ 46%] PASSED test_file_cluster/test.py::test_count test_file_cluster/test.py::test_format_detection [gw9] [ 47%] PASSED test_file_cluster/test.py::test_format_detection [gw0] [ 48%] PASSED test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability [gw8] [ 49%] PASSED test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_simple[hashed] [gw3] [ 50%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_direct-False-False] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_hashed-False-False] [gw1] [ 51%] PASSED test_concurrent_queries_for_user_restriction/test.py::test_exception_message test_backward_compatibility/test_short_strings_aggregation.py::test_backward_compatability test_config_yaml_merge_keys/test.py::test_yaml_merge_keys_conf test_cleanup_after_start/test.py::test_old_dirs_cleanup test_config_xml_main/test.py::test_xml_main_conf [gw5] [ 52%] FAILED test_attach_partition_using_copy/test.py::test_all_replicated test_attach_partition_using_copy/test.py::test_both_mergetree [gw4] [ 53%] PASSED test_backward_compatibility/test_ip_types_binary_compatibility.py::test_ip_types_binary_compatibility test_cluster_discovery/test_auxiliary_keeper.py::test_cluster_discovery_with_auxiliary_keeper_startup_and_stop [gw2] [ 54%] PASSED test_backup_restore_azure_blob_storage/test.py::test_backup_restore test_backup_restore_azure_blob_storage/test.py::test_backup_restore_correct_block_ids [gw0] [ 55%] PASSED test_backward_compatibility/test_short_strings_aggregation.py::test_backward_compatability [gw1] [ 56%] PASSED test_config_xml_main/test.py::test_xml_main_conf test_ddl_worker_replicas/test.py::test_ddl_worker_replicas [gw6] [ 57%] PASSED test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas [gw9] [ 58%] PASSED test_config_yaml_merge_keys/test.py::test_yaml_merge_keys_conf test_cluster_discovery/test_password.py::test_connect_with_password [gw2] [ 59%] PASSED test_backup_restore_azure_blob_storage/test.py::test_backup_restore_correct_block_ids test_backup_restore_azure_blob_storage/test.py::test_backup_restore_diff_container [gw2] [ 60%] PASSED test_backup_restore_azure_blob_storage/test.py::test_backup_restore_diff_container test_backup_restore_azure_blob_storage/test.py::test_backup_restore_on_merge_tree [gw2] [ 61%] PASSED test_backup_restore_azure_blob_storage/test.py::test_backup_restore_on_merge_tree test_backup_restore_azure_blob_storage/test.py::test_backup_restore_with_named_collection_azure_conf1 [gw8] [ 62%] PASSED test_cleanup_after_start/test.py::test_old_dirs_cleanup [gw2] [ 63%] PASSED test_backup_restore_azure_blob_storage/test.py::test_backup_restore_with_named_collection_azure_conf1 test_backup_restore_azure_blob_storage/test.py::test_backup_restore_with_named_collection_azure_conf2 [gw2] [ 64%] PASSED test_backup_restore_azure_blob_storage/test.py::test_backup_restore_with_named_collection_azure_conf2 test_config_reloader_interval/test.py::test_reload_config [gw7] [ 65%] FAILED test_cow_policy/test.py::test_cow_policy[cow_policy_multi_disk] test_cow_policy/test.py::test_cow_policy[cow_policy_multi_volume] [gw8] [ 66%] PASSED test_config_reloader_interval/test.py::test_reload_config [gw1] [ 67%] FAILED test_ddl_worker_replicas/test.py::test_ddl_worker_replicas [gw0] [ 68%] PASSED test_cluster_discovery/test_password.py::test_connect_with_password [gw3] [ 69%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_hashed-False-False] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_ranged[range_hashed-False-False] [gw3] [ 70%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_ranged[range_hashed-False-False] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[cache-False-False] [gw4] [ 71%] PASSED test_cluster_discovery/test_auxiliary_keeper.py::test_cluster_discovery_with_auxiliary_keeper_startup_and_stop test_compression_nested_columns/test.py::test_nested_compression_codec [gw3] [ 72%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[cache-False-False] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[direct-False-False] [gw4] [ 73%] PASSED test_compression_nested_columns/test.py::test_nested_compression_codec [gw6] [ 74%] PASSED test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool [gw6] [ 75%] PASSED test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool [gw5] [ 76%] FAILED test_attach_partition_using_copy/test.py::test_both_mergetree test_attach_partition_using_copy/test.py::test_not_work_on_different_disk [gw3] [ 77%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[direct-False-False] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[flat-False-False] [gw7] [ 78%] FAILED test_cow_policy/test.py::test_cow_policy[cow_policy_multi_volume] test_distributed_directory_monitor_split_batch_on_failure/test.py::test_distributed_background_insert_split_batch_on_failure_OFF [gw3] [ 79%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[flat-False-False] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[hashed-False-False] [gw3] [ 80%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[hashed-False-False] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[cache-False-True] [gw7] [ 81%] PASSED test_distributed_directory_monitor_split_batch_on_failure/test.py::test_distributed_background_insert_split_batch_on_failure_OFF test_distributed_directory_monitor_split_batch_on_failure/test.py::test_distributed_background_insert_split_batch_on_failure_ON [gw5] [ 82%] FAILED test_attach_partition_using_copy/test.py::test_not_work_on_different_disk test_attach_partition_using_copy/test.py::test_only_destination_replicated [gw3] [ 83%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[cache-False-True] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[direct-False-True] [gw7] [ 84%] PASSED test_distributed_directory_monitor_split_batch_on_failure/test.py::test_distributed_background_insert_split_batch_on_failure_ON [gw3] [ 85%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[direct-False-True] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[flat-False-True] [gw3] [ 86%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[flat-False-True] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[hashed-False-True] [gw5] [ 87%] FAILED test_attach_partition_using_copy/test.py::test_only_destination_replicated [gw3] [ 88%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[hashed-False-True] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_cache-True-False] [gw3] [ 89%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_cache-True-False] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_direct-True-False] [gw3] [ 90%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_direct-True-False] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_hashed-True-False] [gw3] [ 91%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_hashed-True-False] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_ranged[range_hashed-True-False] [gw3] [ 92%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_ranged[range_hashed-True-False] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[cache-True-False] [gw3] [ 93%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[cache-True-False] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[direct-True-False] [gw3] [ 94%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[direct-True-False] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[flat-True-False] [gw3] [ 95%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[flat-True-False] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[hashed-True-False] [gw3] [ 96%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[hashed-True-False] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[cache-True-True] [gw3] [ 97%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[cache-True-True] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[direct-True-True] [gw3] [ 98%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[direct-True-True] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[flat-True-True] [gw3] [ 99%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[flat-True-True] test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[hashed-True-True] [gw3] [100%] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[hashed-True-True] =================================== FAILURES =================================== _____________________________ test_all_replicated ______________________________ [gw5] linux -- Python 3.10.12 /usr/bin/python3 start_cluster = def test_all_replicated(start_cluster): cleanup([replica1, replica2]) > create_source_table(replica1, "source", True) test_attach_partition_using_copy/test.py:128: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ test_attach_partition_using_copy/test.py:40: in create_source_table node.query_with_retry( _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = sql = "\n ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7'\n (\n price UInt32,\n ...disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/')\n " stdin = None, timeout = 60, settings = None, user = None, password = None database = None, host = None, ignore_error = False, retry_count = 3 sleep_time = 0.5 check_callback = at 0x7fb883bb5630> parse = False def query_with_retry( self, sql, stdin=None, timeout=None, settings=None, user=None, password=None, database=None, host=None, ignore_error=False, retry_count=20, sleep_time=0.5, check_callback=lambda x: True, parse=False, ): # logging.debug(f"Executing query {sql} on {self.name}") result = None exception_msg = "" for i in range(retry_count): try: result = self.query( sql, stdin=stdin, timeout=timeout, settings=settings, user=user, password=password, database=database, host=host, ignore_error=ignore_error, parse=parse, ) if check_callback(result): return result time.sleep(sleep_time) except QueryRuntimeException as ex: exception_msg = f"{type(ex).__name__}: {str(ex)}" # Container is down, this is likely due to server crash. if "No route to host" in str(ex): raise time.sleep(sleep_time) except Exception as ex: # logging.debug("Retry {} got exception {}".format(i + 1, ex)) exception_msg = f"{type(ex).__name__}: {str(ex)}" time.sleep(sleep_time) if result is not None: return result > raise Exception(f"Can't execute query {sql}\n{exception_msg}") E Exception: Can't execute query E ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' E ( E price UInt32, E date Date, E postcode1 LowCardinality(String), E postcode2 LowCardinality(String), E type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), E is_new UInt8, E duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), E addr1 String, E addr2 String, E street LowCardinality(String), E locality LowCardinality(String), E town LowCardinality(String), E district LowCardinality(String), E county LowCardinality(String) E ) E ENGINE = ReplicatedMergeTree('/clickhouse/tables/1/source', 'replica1') E ORDER BY (postcode1, postcode2, addr1, addr2) E SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') E E QueryRuntimeException: Client failed! Return code: 198, stderr: Received exception from server (version 25.3.8): E Code: 198. DB::Exception: Received from 172.16.2.5:9000. DB::NetException. DB::NetException: Not found address of host: raw.githubusercontent.com: while loading disk metadata. Stack trace: E E 0. ./contrib/llvm-project/libcxx/include/__exception/exception.h:113: Poco::Exception::Exception(String const&, int) @ 0x0000000036f39bb1 E 1. ./build_docker/./src/Common/Exception.cpp:108: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001b1bca51 E 2. ./src/Common/Exception.h:112: DB::NetException::NetException(int, FormatStringHelperImpl::type>, String const&) @ 0x000000001b14702f E 3. ./build_docker/./src/Common/DNSResolver.cpp:113: DB::(anonymous namespace)::hostByName(String const&) @ 0x000000001b1401de E 4. ./build_docker/./src/Common/DNSResolver.cpp:138: DB::DNSResolver::getResolvedIPAdressessWithFiltering(String const&) @ 0x000000001b13d168 E 5. ./build_docker/./src/Common/DNSResolver.cpp:256: DB::DNSResolver::resolveIPAddressWithCache(String const&) @ 0x000000001b13dba0 E 6. ./build_docker/./src/Common/DNSResolver.cpp:276: DB::DNSResolver::resolveHostAllInOriginOrder(String const&) @ 0x000000001b13e8e7 E 7. ./build_docker/./src/Common/HostResolvePool.cpp:54: std::vector> std::__function::__policy_invoker> (String const&)>::__call_impl[abi:ne190107]> (String const&)>>(std::__function::__policy_storage const*, String const&) @ 0x000000001b9595a0 E 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001b955d21 E 9. ./build_docker/./src/Common/HostResolvePool.cpp:66: DB::HostResolver::HostResolver(std::function> (String const&)>&&, String, Poco::Timespan) @ 0x000000001b9557ed E 10. ./build_docker/./src/Common/HostResolvePool.cpp:53: DB::HostResolver::HostResolver(String, Poco::Timespan) @ 0x000000001b9551b4 E 11. ./src/Common/HostResolvePool.h:62: std::shared_ptr DB::HostResolver::create(String const&)::make_shared_enabler::make_shared_enabler(String const&) @ 0x000000001b95ea10 E 12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:41: std::shared_ptr DB::HostResolver::create(String const&)::make_shared_enabler> std::allocate_shared[abi:ne190107] DB::HostResolver::create(String const&)::make_shared_enabler, std::allocator DB::HostResolver::create(String const&)::make_shared_enabler>, String const&, 0>(std::allocator DB::HostResolver::create(String const&)::make_shared_enabler> const&, String const&) @ 0x000000001b95e58d E 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:851: DB::HostResolversPool::getResolver(String const&) @ 0x000000001b95938b E 14. ./build_docker/./src/Common/HTTPConnectionPool.cpp:671: DB::EndpointConnectionPool::prepareNewConnection(DB::ConnectionTimeouts const&, unsigned long*) @ 0x000000001b940b28 E 15. ./build_docker/./src/Common/HTTPConnectionPool.cpp:590: DB::EndpointConnectionPool::getConnection(DB::ConnectionTimeouts const&, unsigned long*) @ 0x000000001b93eeb0 E 16. ./build_docker/./src/IO/HTTPCommon.cpp:63: DB::makeHTTPSession(DB::HTTPConnectionGroupType, Poco::URI const&, DB::ConnectionTimeouts const&, DB::ProxyConfiguration const&, unsigned long*) @ 0x000000001b969334 E 17. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:267: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x00000000206707b2 E 18. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:285: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x0000000020670e3c E 19. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:408: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000020671ebb E 20. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:472: void std::__function::__policy_invoker::__call_impl[abi:ne190107]>(std::__function::__policy_storage const*) @ 0x00000000206777d8 E 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002066c831 E 22. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:465: DB::ReadWriteBufferFromHTTP::nextImpl() @ 0x00000000206744e3 E 23. DB::ReadBuffer::next() @ 0x000000000ba328eb E 24. ./src/IO/ReadBuffer.h:96: DB::WebObjectStorage::loadFiles(String const&, std::unique_lock const&) const @ 0x0000000027a861c2 E 25. ./build_docker/./src/Disks/ObjectStorages/Web/WebObjectStorage.cpp:225: DB::WebObjectStorage::tryGetFileInfo(String const&) const @ 0x0000000027a8ab1f E 26. ./build_docker/./src/Disks/ObjectStorages/Web/WebObjectStorage.cpp:185: DB::WebObjectStorage::tryGetFileInfo(String const&) const @ 0x0000000027a8a450 E 27. ./build_docker/./src/Disks/ObjectStorages/Web/MetadataStorageFromStaticFilesWebServer.cpp:106: DB::MetadataStorageFromStaticFilesWebServer::getStorageObjectsIfExist(String const&) const @ 0x0000000027a82586 E 28. ./build_docker/./src/Disks/ObjectStorages/DiskObjectStorage.cpp:785: DB::DiskObjectStorage::readFileIfExists(String const&, DB::ReadSettings const&, std::optional, std::optional) const @ 0x000000002798e67e E 29. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:380: DB::MergeTreeData::initializeDirectoriesAndFormatVersion(String const&, bool, String const&, bool) @ 0x000000002e338f90 E 30. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:414: DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(DB::TableZnodeInfo const&, DB::LoadingStrictnessLevel, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, std::shared_ptr, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr>, bool, DB::ZooKeeperRetriesInfo const&) @ 0x000000002d8de85c E 31. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:41: std::shared_ptr std::allocate_shared[abi:ne190107], DB::TableZnodeInfo&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>, bool&, DB::ZooKeeperRetriesInfo&, 0>(std::allocator const&, DB::TableZnodeInfo&, DB::LoadingStrictnessLevel const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>&&, bool&, DB::ZooKeeperRetriesInfo&) @ 0x000000002eaf01e1 E . (DNS_ERROR) E (query: ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' E ( E price UInt32, E date Date, E postcode1 LowCardinality(String), E postcode2 LowCardinality(String), E type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), E is_new UInt8, E duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), E addr1 String, E addr2 String, E street LowCardinality(String), E locality LowCardinality(String), E town LowCardinality(String), E district LowCardinality(String), E county LowCardinality(String) E ) E ENGINE = ReplicatedMergeTree('/clickhouse/tables/1/source', 'replica1') E ORDER BY (postcode1, postcode2, addr1, addr2) E SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') E ) helpers/cluster.py:3722: Exception ---------------------------- Captured stdout setup ----------------------------- Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml ---------------------------- Captured stderr setup ----------------------------- Command:[docker ps | wc -l] Stdout:1 No running containers Pruning Docker networks Command:[docker network prune --force] Command:[sysctl net.ipv4.ip_local_port_range='55000 65535'] Stdout:net.ipv4.ip_local_port_range = 55000 65535 Running tests in /ClickHouse/tests/integration/test_attach_partition_using_copy/test.py Cluster start called. is_up=False Docker networks for project roottestattachpartitionusingcopy-gw5 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestattachpartitionusingcopy-gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestattachpartitionusingcopy-gw5 are DRIVER VOLUME NAME Cleanup called Docker networks for project roottestattachpartitionusingcopy-gw5 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestattachpartitionusingcopy-gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestattachpartitionusingcopy-gw5 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestattachpartitionusingcopy-gw5-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestattachpartitionusingcopy-gw5 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stderr:Error response from daemon: a prune operation is already running Exitcode:1 Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:1 Volumes pruned: 1 Setup directory for instance: replica1 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_attach_partition_using_copy/configs/remote_servers.xml'] to /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/configs/config.d Setup database dir /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/database Setup logs dir /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Setup directory for instance: replica2 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_attach_partition_using_copy/configs/remote_servers.xml'] to /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/configs/config.d Setup database dir /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/database Setup logs dir /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'LSAN_OPTIONS': 'suppressions=/etc/clickhouse-server/lsan_suppressions.txt', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:5ccda723c1fc', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/.env Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/.env --project-name roottestattachpartitionusingcopy-gw5 --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/docker-compose.yml pull] Stderr: replica1 Skipped - Image is already being pulled by replica2 Stderr: zoo1 Skipped - Image is already being pulled by replica2 Stderr: zoo2 Skipped - Image is already being pulled by replica2 Stderr: zoo3 Skipped - Image is already being pulled by replica2 Stderr: replica2 Pulling Stderr: replica2 Pulled Setup ZooKeeper Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper1/log', '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper1/config', '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper1/coordination', '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper2/log', '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper2/config', '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper2/coordination', '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper3/log', '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper3/config', '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper3/coordination'] Command:[docker compose --project-name roottestattachpartitionusingcopy-gw5 --env-file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --verbose up -d] Stderr:time="2026-04-10T18:13:56Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Network roottestattachpartitionusingcopy-gw5_default Creating Stderr: Network roottestattachpartitionusingcopy-gw5_default Created Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Creating Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Creating Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Creating Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Created Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Created Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Created Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Starting Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Starting Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Starting Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Started Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Started Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Started Stderr:time="2026-04-10T18:13:58Z" level=debug msg="otel error" error="" Stderr:time="2026-04-10T18:13:58Z" level=debug msg="otel error" error="" Wait ZooKeeper to start get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestattachpartitionusingcopy-gw5-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.2.2, port:2181, use_ssl:False Connecting to 172.16.2.2(172.16.2.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.2.2(172.16.2.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.2.2(172.16.2.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.2.2(172.16.2.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.2.2(172.16.2.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.2.2(172.16.2.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestattachpartitionusingcopy-gw5-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.2.4, port:2181, use_ssl:False Connecting to 172.16.2.4(172.16.2.4):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestattachpartitionusingcopy-gw5-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.2.3, port:2181, use_ssl:False Connecting to 172.16.2.3(172.16.2.3):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/.env --project-name roottestattachpartitionusingcopy-gw5 --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/.env --project-name roottestattachpartitionusingcopy-gw5 --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/docker-compose.yml up -d --no-recreate] Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Running Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Running Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Running Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Creating Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Creating Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Created Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Created Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Starting Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Starting Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Started Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Started ClickHouse instance created get_instance_ip instance_name=replica1 http://localhost:None "GET /v1.46/containers/roottestattachpartitionusingcopy-gw5-replica1-1/json HTTP/1.1" 200 None get_instance_ip instance_name=replica1 http://localhost:None "GET /v1.46/containers/roottestattachpartitionusingcopy-gw5-replica1-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in replica1, ip: 172.16.2.5... http://localhost:None "GET /v1.46/containers/roottestattachpartitionusingcopy-gw5-replica1-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None ClickHouse replica1 started get_instance_ip instance_name=replica2 http://localhost:None "GET /v1.46/containers/roottestattachpartitionusingcopy-gw5-replica2-1/json HTTP/1.1" 200 None get_instance_ip instance_name=replica2 http://localhost:None "GET /v1.46/containers/roottestattachpartitionusingcopy-gw5-replica2-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in replica2, ip: 172.16.2.6... http://localhost:None "GET /v1.46/containers/roottestattachpartitionusingcopy-gw5-replica2-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/7ee32c09a617540b5b80cdabc7d2d51a6472107932e6e395ab1f4b79f99b2732/json HTTP/1.1" 200 None ClickHouse replica2 started ------------------------------ Captured log setup ------------------------------ 2026-04-10 18:13:45.870000 [ 618 ] DEBUG : Command:[docker ps | wc -l] (cluster.py:121, run_and_check) 2026-04-10 18:13:45.915000 [ 618 ] DEBUG : Stdout:1 (cluster.py:145, run_and_check) 2026-04-10 18:13:45.916000 [ 618 ] DEBUG : No running containers (conftest.py:95, cleanup_environment) 2026-04-10 18:13:45.916000 [ 618 ] DEBUG : Pruning Docker networks (conftest.py:97, cleanup_environment) 2026-04-10 18:13:45.916000 [ 618 ] DEBUG : Command:[docker network prune --force] (cluster.py:121, run_and_check) 2026-04-10 18:13:45.946000 [ 618 ] DEBUG : Command:[sysctl net.ipv4.ip_local_port_range='55000 65535'] (cluster.py:121, run_and_check) 2026-04-10 18:13:45.950000 [ 618 ] DEBUG : Stdout:net.ipv4.ip_local_port_range = 55000 65535 (cluster.py:145, run_and_check) 2026-04-10 18:13:45.951000 [ 618 ] INFO : Running tests in /ClickHouse/tests/integration/test_attach_partition_using_copy/test.py (cluster.py:2748, start) 2026-04-10 18:13:45.951000 [ 618 ] DEBUG : Cluster start called. is_up=False (cluster.py:2755, start) 2026-04-10 18:13:45.992000 [ 618 ] DEBUG : Docker networks for project roottestattachpartitionusingcopy-gw5 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 18:13:46.027000 [ 618 ] DEBUG : Docker containers for project roottestattachpartitionusingcopy-gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 18:13:46.062000 [ 618 ] DEBUG : Docker volumes for project roottestattachpartitionusingcopy-gw5 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 18:13:46.062000 [ 618 ] DEBUG : Cleanup called (cluster.py:861, cleanup) 2026-04-10 18:13:46.102000 [ 618 ] DEBUG : Docker networks for project roottestattachpartitionusingcopy-gw5 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 18:13:46.152000 [ 618 ] DEBUG : Docker containers for project roottestattachpartitionusingcopy-gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 18:13:46.200000 [ 618 ] DEBUG : Docker volumes for project roottestattachpartitionusingcopy-gw5 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 18:13:46.200000 [ 618 ] DEBUG : Command:[docker container list --all --filter name='^/roottestattachpartitionusingcopy-gw5-.*-1$' --format '{{.ID}}:{{.Names}}'] (cluster.py:121, run_and_check) 2026-04-10 18:13:46.237000 [ 618 ] DEBUG : Unstopped containers: {} (cluster.py:875, cleanup) 2026-04-10 18:13:46.237000 [ 618 ] DEBUG : No running containers for project: roottestattachpartitionusingcopy-gw5 (cluster.py:889, cleanup) 2026-04-10 18:13:46.237000 [ 618 ] DEBUG : Trying to prune unused networks... (cluster.py:895, cleanup) 2026-04-10 18:13:46.266000 [ 618 ] DEBUG : Trying to prune unused images... (cluster.py:911, cleanup) 2026-04-10 18:13:46.266000 [ 618 ] DEBUG : Command:[docker image prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:13:46.291000 [ 618 ] DEBUG : Stderr:Error response from daemon: a prune operation is already running (cluster.py:147, run_and_check) 2026-04-10 18:13:46.291000 [ 618 ] DEBUG : Exitcode:1 (cluster.py:149, run_and_check) 2026-04-10 18:13:46.292000 [ 618 ] DEBUG : Trying to prune unused volumes... (cluster.py:920, cleanup) 2026-04-10 18:13:46.292000 [ 618 ] DEBUG : Command:[docker volume ls | wc -l] (cluster.py:121, run_and_check) 2026-04-10 18:13:46.322000 [ 618 ] DEBUG : Stdout:1 (cluster.py:145, run_and_check) 2026-04-10 18:13:46.323000 [ 618 ] DEBUG : Volumes pruned: 1 (cluster.py:925, cleanup) 2026-04-10 18:13:46.323000 [ 618 ] DEBUG : Setup directory for instance: replica1 (cluster.py:2768, start) 2026-04-10 18:13:46.324000 [ 618 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 18:13:46.324000 [ 618 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 18:13:46.324000 [ 618 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 18:13:46.325000 [ 618 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 18:13:46.326000 [ 618 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_attach_partition_using_copy/configs/remote_servers.xml'] to /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 18:13:46.327000 [ 618 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/database (cluster.py:4772, create_dir) 2026-04-10 18:13:46.327000 [ 618 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/logs (cluster.py:4783, create_dir) 2026-04-10 18:13:46.327000 [ 618 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4864, create_dir) 2026-04-10 18:13:46.327000 [ 618 ] DEBUG : Setup directory for instance: replica2 (cluster.py:2768, start) 2026-04-10 18:13:46.328000 [ 618 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 18:13:46.328000 [ 618 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 18:13:46.328000 [ 618 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 18:13:46.329000 [ 618 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 18:13:46.330000 [ 618 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_attach_partition_using_copy/configs/remote_servers.xml'] to /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 18:13:46.330000 [ 618 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/database (cluster.py:4772, create_dir) 2026-04-10 18:13:46.330000 [ 618 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/logs (cluster.py:4783, create_dir) 2026-04-10 18:13:46.330000 [ 618 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4864, create_dir) 2026-04-10 18:13:46.331000 [ 618 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'LSAN_OPTIONS': 'suppressions=/etc/clickhouse-server/lsan_suppressions.txt', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:5ccda723c1fc', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/.env (cluster.py:96, _create_env_file) 2026-04-10 18:13:46.331000 [ 618 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-04-10 18:13:46.332000 [ 618 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-04-10 18:13:46.332000 [ 618 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-04-10 18:13:46.332000 [ 618 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-04-10 18:13:46.352000 [ 618 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 826 (connectionpool.py:547, _make_request) 2026-04-10 18:13:46.353000 [ 618 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/.env --project-name roottestattachpartitionusingcopy-gw5 --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/docker-compose.yml pull] (cluster.py:121, run_and_check) 2026-04-10 18:13:56.858000 [ 618 ] DEBUG : Stderr: replica1 Skipped - Image is already being pulled by replica2 (cluster.py:147, run_and_check) 2026-04-10 18:13:56.859000 [ 618 ] DEBUG : Stderr: zoo1 Skipped - Image is already being pulled by replica2 (cluster.py:147, run_and_check) 2026-04-10 18:13:56.859000 [ 618 ] DEBUG : Stderr: zoo2 Skipped - Image is already being pulled by replica2 (cluster.py:147, run_and_check) 2026-04-10 18:13:56.859000 [ 618 ] DEBUG : Stderr: zoo3 Skipped - Image is already being pulled by replica2 (cluster.py:147, run_and_check) 2026-04-10 18:13:56.859000 [ 618 ] DEBUG : Stderr: replica2 Pulling (cluster.py:147, run_and_check) 2026-04-10 18:13:56.859000 [ 618 ] DEBUG : Stderr: replica2 Pulled (cluster.py:147, run_and_check) 2026-04-10 18:13:56.860000 [ 618 ] DEBUG : Setup ZooKeeper (cluster.py:2809, start) 2026-04-10 18:13:56.860000 [ 618 ] DEBUG : Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper1/log', '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper1/config', '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper1/coordination', '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper2/log', '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper2/config', '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper2/coordination', '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper3/log', '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper3/config', '/ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/keeper3/coordination'] (cluster.py:2810, start) 2026-04-10 18:13:56.861000 [ 618 ] DEBUG : Command:[docker compose --project-name roottestattachpartitionusingcopy-gw5 --env-file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --verbose up -d] (cluster.py:121, run_and_check) 2026-04-10 18:13:58.429000 [ 618 ] DEBUG : Stderr:time="2026-04-10T18:13:56Z" level=trace msg="Docker Desktop integration not enabled" (cluster.py:147, run_and_check) 2026-04-10 18:13:58.429000 [ 618 ] DEBUG : Stderr: Network roottestattachpartitionusingcopy-gw5_default Creating (cluster.py:147, run_and_check) 2026-04-10 18:13:58.430000 [ 618 ] DEBUG : Stderr: Network roottestattachpartitionusingcopy-gw5_default Created (cluster.py:147, run_and_check) 2026-04-10 18:13:58.430000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:13:58.430000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:13:58.430000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:13:58.430000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:13:58.430000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:13:58.430000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:13:58.430000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:13:58.431000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:13:58.431000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:13:58.431000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:13:58.431000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:13:58.431000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:13:58.431000 [ 618 ] DEBUG : Stderr:time="2026-04-10T18:13:58Z" level=debug msg="otel error" error="" (cluster.py:147, run_and_check) 2026-04-10 18:13:58.431000 [ 618 ] DEBUG : Stderr:time="2026-04-10T18:13:58Z" level=debug msg="otel error" error="" (cluster.py:147, run_and_check) 2026-04-10 18:13:58.431000 [ 618 ] DEBUG : Wait ZooKeeper to start (cluster.py:2446, wait_zookeeper_to_start) 2026-04-10 18:13:58.431000 [ 618 ] DEBUG : get_instance_ip instance_name=zoo1 (cluster.py:2015, get_instance_ip) 2026-04-10 18:13:58.436000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestattachpartitionusingcopy-gw5-zoo1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:13:58.437000 [ 618 ] DEBUG : get_kazoo_client: zoo1, ip:172.16.2.2, port:2181, use_ssl:False (cluster.py:3322, get_kazoo_client) 2026-04-10 18:13:58.439000 [ 618 ] INFO : Connecting to 172.16.2.2(172.16.2.2):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 18:13:58.439000 [ 618 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-04-10 18:13:58.571000 [ 618 ] INFO : Connecting to 172.16.2.2(172.16.2.2):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 18:13:58.572000 [ 618 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-04-10 18:13:58.881000 [ 618 ] INFO : Connecting to 172.16.2.2(172.16.2.2):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 18:13:58.883000 [ 618 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-04-10 18:13:59.380000 [ 618 ] INFO : Connecting to 172.16.2.2(172.16.2.2):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 18:13:59.381000 [ 618 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-04-10 18:14:00.292000 [ 618 ] INFO : Connecting to 172.16.2.2(172.16.2.2):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 18:14:00.294000 [ 618 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-04-10 18:14:01.795000 [ 618 ] INFO : Connecting to 172.16.2.2(172.16.2.2):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 18:14:01.796000 [ 618 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-04-10 18:14:01.817000 [ 618 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-04-10 18:14:01.817000 [ 618 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-04-10 18:14:01.819000 [ 618 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-04-10 18:14:01.821000 [ 618 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-04-10 18:14:01.843000 [ 618 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-04-10 18:14:01.844000 [ 618 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-04-10 18:14:01.844000 [ 618 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-04-10 18:14:01.908000 [ 618 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-04-10 18:14:01.910000 [ 618 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-04-10 18:14:01.910000 [ 618 ] DEBUG : get_instance_ip instance_name=zoo2 (cluster.py:2015, get_instance_ip) 2026-04-10 18:14:01.916000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestattachpartitionusingcopy-gw5-zoo2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:01.917000 [ 618 ] DEBUG : get_kazoo_client: zoo2, ip:172.16.2.4, port:2181, use_ssl:False (cluster.py:3322, get_kazoo_client) 2026-04-10 18:14:01.920000 [ 618 ] INFO : Connecting to 172.16.2.4(172.16.2.4):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 18:14:01.921000 [ 618 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-04-10 18:14:01.936000 [ 618 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-04-10 18:14:01.936000 [ 618 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-04-10 18:14:01.939000 [ 618 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-04-10 18:14:01.944000 [ 618 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-04-10 18:14:01.959000 [ 618 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-04-10 18:14:01.959000 [ 618 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-04-10 18:14:01.959000 [ 618 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-04-10 18:14:02.050000 [ 618 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-04-10 18:14:02.051000 [ 618 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-04-10 18:14:02.051000 [ 618 ] DEBUG : get_instance_ip instance_name=zoo3 (cluster.py:2015, get_instance_ip) 2026-04-10 18:14:02.054000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestattachpartitionusingcopy-gw5-zoo3-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:02.055000 [ 618 ] DEBUG : get_kazoo_client: zoo3, ip:172.16.2.3, port:2181, use_ssl:False (cluster.py:3322, get_kazoo_client) 2026-04-10 18:14:02.059000 [ 618 ] INFO : Connecting to 172.16.2.3(172.16.2.3):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 18:14:02.061000 [ 618 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-04-10 18:14:02.068000 [ 618 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-04-10 18:14:02.072000 [ 618 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-04-10 18:14:02.076000 [ 618 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-04-10 18:14:02.077000 [ 618 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-04-10 18:14:02.085000 [ 618 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-04-10 18:14:02.086000 [ 618 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-04-10 18:14:02.086000 [ 618 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-04-10 18:14:02.187000 [ 618 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-04-10 18:14:02.187000 [ 618 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-04-10 18:14:02.187000 [ 618 ] DEBUG : All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') (cluster.py:2462, wait_zookeeper_nodes_to_start) 2026-04-10 18:14:02.188000 [ 618 ] DEBUG : ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/.env --project-name roottestattachpartitionusingcopy-gw5 --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/docker-compose.yml up -d --no-recreate') (cluster.py:3149, start) 2026-04-10 18:14:02.188000 [ 618 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/.env --project-name roottestattachpartitionusingcopy-gw5 --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/docker-compose.yml up -d --no-recreate] (cluster.py:121, run_and_check) 2026-04-10 18:14:03.181000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Running (cluster.py:147, run_and_check) 2026-04-10 18:14:03.181000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Running (cluster.py:147, run_and_check) 2026-04-10 18:14:03.181000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Running (cluster.py:147, run_and_check) 2026-04-10 18:14:03.181000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:14:03.181000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:14:03.182000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:14:03.182000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:14:03.182000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:14:03.182000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:14:03.182000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:14:03.182000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:14:03.182000 [ 618 ] DEBUG : ClickHouse instance created (cluster.py:3157, start) 2026-04-10 18:14:03.183000 [ 618 ] DEBUG : get_instance_ip instance_name=replica1 (cluster.py:2015, get_instance_ip) 2026-04-10 18:14:03.185000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestattachpartitionusingcopy-gw5-replica1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:03.186000 [ 618 ] DEBUG : get_instance_ip instance_name=replica1 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:14:03.189000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestattachpartitionusingcopy-gw5-replica1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:03.189000 [ 618 ] DEBUG : Waiting for ClickHouse start in replica1, ip: 172.16.2.5... (cluster.py:3165, start) 2026-04-10 18:14:03.194000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestattachpartitionusingcopy-gw5-replica1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:03.197000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:03.301000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:03.413000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:03.526000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:03.631000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:03.737000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:03.843000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:03.948000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:04.052000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:04.157000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:04.264000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:04.378000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:04.490000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:04.603000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:04.710000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:04.816000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:04.924000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:05.028000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:05.134000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:05.240000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:05.361000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:05.468000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:05.572000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:05.676000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:05.781000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:05.886000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:05.992000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:06.098000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:06.205000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/81755631cce0a31227cf6f0f99b7384228df0c62c0106f9fe990d032b8acd9de/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:06.206000 [ 618 ] DEBUG : ClickHouse replica1 started (cluster.py:3169, start) 2026-04-10 18:14:06.206000 [ 618 ] DEBUG : get_instance_ip instance_name=replica2 (cluster.py:2015, get_instance_ip) 2026-04-10 18:14:06.209000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestattachpartitionusingcopy-gw5-replica2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:06.210000 [ 618 ] DEBUG : get_instance_ip instance_name=replica2 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:14:06.213000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestattachpartitionusingcopy-gw5-replica2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:06.214000 [ 618 ] DEBUG : Waiting for ClickHouse start in replica2, ip: 172.16.2.6... (cluster.py:3165, start) 2026-04-10 18:14:06.216000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestattachpartitionusingcopy-gw5-replica2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:06.220000 [ 618 ] DEBUG : http://localhost:None "GET /v1.46/containers/7ee32c09a617540b5b80cdabc7d2d51a6472107932e6e395ab1f4b79f99b2732/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:06.221000 [ 618 ] DEBUG : ClickHouse replica2 started (cluster.py:3169, start) ----------------------------- Captured stderr call ----------------------------- Executing query DROP TABLE IF EXISTS source SYNC on replica1 Executing query DROP TABLE IF EXISTS destination SYNC on replica1 Executing query DROP TABLE IF EXISTS source SYNC on replica2 Executing query DROP TABLE IF EXISTS destination SYNC on replica2 Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/1/source', 'replica1') ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/1/source', 'replica1') ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/1/source', 'replica1') ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 ------------------------------ Captured log call ------------------------------- 2026-04-10 18:14:06.224000 [ 618 ] DEBUG : Executing query DROP TABLE IF EXISTS source SYNC on replica1 (cluster.py:3658, query) 2026-04-10 18:14:06.699000 [ 618 ] DEBUG : Executing query DROP TABLE IF EXISTS destination SYNC on replica1 (cluster.py:3658, query) 2026-04-10 18:14:07.069000 [ 618 ] DEBUG : Executing query DROP TABLE IF EXISTS source SYNC on replica2 (cluster.py:3658, query) 2026-04-10 18:14:07.442000 [ 618 ] DEBUG : Executing query DROP TABLE IF EXISTS destination SYNC on replica2 (cluster.py:3658, query) 2026-04-10 18:14:07.912000 [ 618 ] DEBUG : Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/1/source', 'replica1') ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 (cluster.py:3658, query) 2026-04-10 18:15:02.950000 [ 618 ] DEBUG : Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/1/source', 'replica1') ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 (cluster.py:3658, query) 2026-04-10 18:15:58.650000 [ 618 ] DEBUG : Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/1/source', 'replica1') ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 (cluster.py:3658, query) ____________________ test_cow_policy[cow_policy_multi_disk] ____________________ [gw7] linux -- Python 3.10.12 /usr/bin/python3 start_cluster = storage_policy = 'cow_policy_multi_disk' @pytest.mark.parametrize("storage_policy", ["cow_policy_multi_disk", "cow_policy_multi_volume"]) def test_cow_policy(start_cluster, storage_policy): try: > node.query_with_retry( f""" ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS storage_policy = '{storage_policy}' """, timeout=60, retry_count=3, ) test_cow_policy/test.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = sql = "\n ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7'\n (\n ...R BY (postcode1, postcode2, addr1, addr2)\n SETTINGS storage_policy = 'cow_policy_multi_disk'\n " stdin = None, timeout = 60, settings = None, user = None, password = None database = None, host = None, ignore_error = False, retry_count = 3 sleep_time = 0.5 check_callback = at 0x7f5b38871630> parse = False def query_with_retry( self, sql, stdin=None, timeout=None, settings=None, user=None, password=None, database=None, host=None, ignore_error=False, retry_count=20, sleep_time=0.5, check_callback=lambda x: True, parse=False, ): # logging.debug(f"Executing query {sql} on {self.name}") result = None exception_msg = "" for i in range(retry_count): try: result = self.query( sql, stdin=stdin, timeout=timeout, settings=settings, user=user, password=password, database=database, host=host, ignore_error=ignore_error, parse=parse, ) if check_callback(result): return result time.sleep(sleep_time) except QueryRuntimeException as ex: exception_msg = f"{type(ex).__name__}: {str(ex)}" # Container is down, this is likely due to server crash. if "No route to host" in str(ex): raise time.sleep(sleep_time) except Exception as ex: # logging.debug("Retry {} got exception {}".format(i + 1, ex)) exception_msg = f"{type(ex).__name__}: {str(ex)}" time.sleep(sleep_time) if result is not None: return result > raise Exception(f"Can't execute query {sql}\n{exception_msg}") E Exception: Can't execute query E ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' E ( E price UInt32, E date Date, E postcode1 LowCardinality(String), E postcode2 LowCardinality(String), E type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), E is_new UInt8, E duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), E addr1 String, E addr2 String, E street LowCardinality(String), E locality LowCardinality(String), E town LowCardinality(String), E district LowCardinality(String), E county LowCardinality(String) E ) E ENGINE = MergeTree E ORDER BY (postcode1, postcode2, addr1, addr2) E SETTINGS storage_policy = 'cow_policy_multi_disk' E E QueryTimeoutExceedException: Client timed out! helpers/cluster.py:3722: Exception ---------------------------- Captured stdout setup ----------------------------- Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml ---------------------------- Captured stderr setup ----------------------------- Running tests in /ClickHouse/tests/integration/test_cow_policy/test.py Cluster start called. is_up=False Docker networks for project roottestcowpolicy-gw7 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestcowpolicy-gw7 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestcowpolicy-gw7 are DRIVER VOLUME NAME Cleanup called Docker networks for project roottestcowpolicy-gw7 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestcowpolicy-gw7 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestcowpolicy-gw7 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestcowpolicy-gw7-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestcowpolicy-gw7 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:9 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 9 Setup directory for instance: node Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_cow_policy/configs/overrides.yaml'] to /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/configs/config.d Setup database dir /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/database Setup logs dir /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'LSAN_OPTIONS': 'suppressions=/etc/clickhouse-server/lsan_suppressions.txt', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw'} stored in /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/.env Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/.env --project-name roottestcowpolicy-gw7 --file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/docker-compose.yml pull] Stderr: node Pulling Stderr: node Pulled ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/.env --project-name roottestcowpolicy-gw7 --file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/.env --project-name roottestcowpolicy-gw7 --file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/docker-compose.yml up -d --no-recreate] Stderr: Network roottestcowpolicy-gw7_default Creating Stderr: Network roottestcowpolicy-gw7_default Created Stderr: Container roottestcowpolicy-gw7-node-1 Creating Stderr: Container roottestcowpolicy-gw7-node-1 Created Stderr: Container roottestcowpolicy-gw7-node-1 Starting Stderr: Container roottestcowpolicy-gw7-node-1 Started ClickHouse instance created get_instance_ip instance_name=node http://localhost:None "GET /v1.46/containers/roottestcowpolicy-gw7-node-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node http://localhost:None "GET /v1.46/containers/roottestcowpolicy-gw7-node-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node, ip: 172.16.9.2... http://localhost:None "GET /v1.46/containers/roottestcowpolicy-gw7-node-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None ClickHouse node started ------------------------------ Captured log setup ------------------------------ 2026-04-10 18:14:21.051000 [ 624 ] INFO : Running tests in /ClickHouse/tests/integration/test_cow_policy/test.py (cluster.py:2748, start) 2026-04-10 18:14:21.051000 [ 624 ] DEBUG : Cluster start called. is_up=False (cluster.py:2755, start) 2026-04-10 18:14:21.100000 [ 624 ] DEBUG : Docker networks for project roottestcowpolicy-gw7 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 18:14:21.138000 [ 624 ] DEBUG : Docker containers for project roottestcowpolicy-gw7 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 18:14:21.186000 [ 624 ] DEBUG : Docker volumes for project roottestcowpolicy-gw7 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 18:14:21.187000 [ 624 ] DEBUG : Cleanup called (cluster.py:861, cleanup) 2026-04-10 18:14:21.219000 [ 624 ] DEBUG : Docker networks for project roottestcowpolicy-gw7 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 18:14:21.258000 [ 624 ] DEBUG : Docker containers for project roottestcowpolicy-gw7 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 18:14:21.325000 [ 624 ] DEBUG : Docker volumes for project roottestcowpolicy-gw7 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 18:14:21.325000 [ 624 ] DEBUG : Command:[docker container list --all --filter name='^/roottestcowpolicy-gw7-.*-1$' --format '{{.ID}}:{{.Names}}'] (cluster.py:121, run_and_check) 2026-04-10 18:14:21.370000 [ 624 ] DEBUG : Unstopped containers: {} (cluster.py:875, cleanup) 2026-04-10 18:14:21.371000 [ 624 ] DEBUG : No running containers for project: roottestcowpolicy-gw7 (cluster.py:889, cleanup) 2026-04-10 18:14:21.371000 [ 624 ] DEBUG : Trying to prune unused networks... (cluster.py:895, cleanup) 2026-04-10 18:14:21.414000 [ 624 ] DEBUG : Trying to prune unused images... (cluster.py:911, cleanup) 2026-04-10 18:14:21.414000 [ 624 ] DEBUG : Command:[docker image prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:14:21.464000 [ 624 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 18:14:21.465000 [ 624 ] DEBUG : Images pruned (cluster.py:914, cleanup) 2026-04-10 18:14:21.465000 [ 624 ] DEBUG : Trying to prune unused volumes... (cluster.py:920, cleanup) 2026-04-10 18:14:21.466000 [ 624 ] DEBUG : Command:[docker volume ls | wc -l] (cluster.py:121, run_and_check) 2026-04-10 18:14:21.517000 [ 624 ] DEBUG : Stdout:9 (cluster.py:145, run_and_check) 2026-04-10 18:14:21.518000 [ 624 ] DEBUG : Command:[docker volume prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:14:21.564000 [ 624 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 18:14:21.565000 [ 624 ] DEBUG : Volumes pruned: 9 (cluster.py:925, cleanup) 2026-04-10 18:14:21.565000 [ 624 ] DEBUG : Setup directory for instance: node (cluster.py:2768, start) 2026-04-10 18:14:21.566000 [ 624 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 18:14:21.566000 [ 624 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 18:14:21.567000 [ 624 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 18:14:21.568000 [ 624 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 18:14:21.569000 [ 624 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_cow_policy/configs/overrides.yaml'] to /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 18:14:21.572000 [ 624 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/database (cluster.py:4772, create_dir) 2026-04-10 18:14:21.573000 [ 624 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/logs (cluster.py:4783, create_dir) 2026-04-10 18:14:21.573000 [ 624 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4864, create_dir) 2026-04-10 18:14:21.574000 [ 624 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'LSAN_OPTIONS': 'suppressions=/etc/clickhouse-server/lsan_suppressions.txt', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw'} stored in /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/.env (cluster.py:96, _create_env_file) 2026-04-10 18:14:21.574000 [ 624 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-04-10 18:14:21.574000 [ 624 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-04-10 18:14:21.575000 [ 624 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-04-10 18:14:21.575000 [ 624 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-04-10 18:14:21.599000 [ 624 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 826 (connectionpool.py:547, _make_request) 2026-04-10 18:14:21.603000 [ 624 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/.env --project-name roottestcowpolicy-gw7 --file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/docker-compose.yml pull] (cluster.py:121, run_and_check) 2026-04-10 18:14:32.149000 [ 624 ] DEBUG : Stderr: node Pulling (cluster.py:147, run_and_check) 2026-04-10 18:14:32.149000 [ 624 ] DEBUG : Stderr: node Pulled (cluster.py:147, run_and_check) 2026-04-10 18:14:32.150000 [ 624 ] DEBUG : ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/.env --project-name roottestcowpolicy-gw7 --file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/docker-compose.yml up -d --no-recreate') (cluster.py:3149, start) 2026-04-10 18:14:32.150000 [ 624 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/.env --project-name roottestcowpolicy-gw7 --file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/docker-compose.yml up -d --no-recreate] (cluster.py:121, run_and_check) 2026-04-10 18:14:33.084000 [ 624 ] DEBUG : Stderr: Network roottestcowpolicy-gw7_default Creating (cluster.py:147, run_and_check) 2026-04-10 18:14:33.084000 [ 624 ] DEBUG : Stderr: Network roottestcowpolicy-gw7_default Created (cluster.py:147, run_and_check) 2026-04-10 18:14:33.085000 [ 624 ] DEBUG : Stderr: Container roottestcowpolicy-gw7-node-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:14:33.085000 [ 624 ] DEBUG : Stderr: Container roottestcowpolicy-gw7-node-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:14:33.085000 [ 624 ] DEBUG : Stderr: Container roottestcowpolicy-gw7-node-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:14:33.085000 [ 624 ] DEBUG : Stderr: Container roottestcowpolicy-gw7-node-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:14:33.085000 [ 624 ] DEBUG : ClickHouse instance created (cluster.py:3157, start) 2026-04-10 18:14:33.085000 [ 624 ] DEBUG : get_instance_ip instance_name=node (cluster.py:2015, get_instance_ip) 2026-04-10 18:14:33.091000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestcowpolicy-gw7-node-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:33.092000 [ 624 ] DEBUG : get_instance_ip instance_name=node (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:14:33.095000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestcowpolicy-gw7-node-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:33.096000 [ 624 ] DEBUG : Waiting for ClickHouse start in node, ip: 172.16.9.2... (cluster.py:3165, start) 2026-04-10 18:14:33.098000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestcowpolicy-gw7-node-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:33.102000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:33.206000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:33.311000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:33.417000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:33.528000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:33.638000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:33.744000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:33.849000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:33.957000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:34.071000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:34.180000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:34.287000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:34.392000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:34.498000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:34.605000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:34.712000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:34.819000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:34.927000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:35.032000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:35.137000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:35.245000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:35.351000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:35.456000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:35.562000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:35.668000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:35.777000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:35.885000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:35.992000 [ 624 ] DEBUG : http://localhost:None "GET /v1.46/containers/2e3ef9aa532ba85c724bb906c80c8eecbd938882e0f1178111e7b4568f7f2177/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:14:35.994000 [ 624 ] DEBUG : ClickHouse node started (cluster.py:3169, start) ----------------------------- Captured stderr call ----------------------------- Executing query ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS storage_policy = 'cow_policy_multi_disk' on node Executing query ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS storage_policy = 'cow_policy_multi_disk' on node Executing query ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS storage_policy = 'cow_policy_multi_disk' on node Timed out. Last stdout:, stderr: Executing query DROP TABLE IF EXISTS uk_price_paid SYNC on node ------------------------------ Captured log call ------------------------------- 2026-04-10 18:14:35.998000 [ 624 ] DEBUG : Executing query ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS storage_policy = 'cow_policy_multi_disk' on node (cluster.py:3658, query) 2026-04-10 18:15:33.077000 [ 624 ] DEBUG : Executing query ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS storage_policy = 'cow_policy_multi_disk' on node (cluster.py:3658, query) 2026-04-10 18:16:33.157000 [ 624 ] DEBUG : Executing query ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS storage_policy = 'cow_policy_multi_disk' on node (cluster.py:3658, query) 2026-04-10 18:17:33.648000 [ 624 ] DEBUG : Timed out. Last stdout:, stderr: (client.py:242, get_answer) 2026-04-10 18:17:34.187000 [ 624 ] DEBUG : Executing query DROP TABLE IF EXISTS uk_price_paid SYNC on node (cluster.py:3658, query) ___________________________ test_ddl_worker_replicas ___________________________ [gw1] linux -- Python 3.10.12 /usr/bin/python3 started_cluster = def test_ddl_worker_replicas(started_cluster): for replica in ["node1:9000", "node2:9000", "node3:9000", "node4:9000"]: # wait until the replicas path is created node1.query_with_retry( sql=f"SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/{replica}'", check_callback=lambda result: result == 1, ) result = node1.query( f"SELECT name, value, ephemeralOwner FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/{replica}'" ).strip() print(f"result: {replica} {result}") lines = list(result.split("\n")) assert len(lines) == 1 parts = list(lines[0].split("\t")) > assert len(parts) == 3 E AssertionError: assert 1 == 3 E + where 1 = len(['']) test_ddl_worker_replicas/test.py:53: AssertionError ---------------------------- Captured stdout setup ----------------------------- Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml ---------------------------- Captured stderr setup ----------------------------- Running tests in /ClickHouse/tests/integration/test_ddl_worker_replicas/test.py Cluster start called. is_up=False Docker networks for project roottestddlworkerreplicas-gw1 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestddlworkerreplicas-gw1 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestddlworkerreplicas-gw1 are DRIVER VOLUME NAME Cleanup called Docker networks for project roottestddlworkerreplicas-gw1 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestddlworkerreplicas-gw1 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestddlworkerreplicas-gw1 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestddlworkerreplicas-gw1-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestddlworkerreplicas-gw1 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:9 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 9 Setup directory for instance: node1 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_ddl_worker_replicas/configs/remote_servers.xml'] to /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/configs/config.d Setup database dir /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/database Setup logs dir /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/logs Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon -- ; coproc tail -f /dev/null; wait $$!" Setup directory for instance: node2 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_ddl_worker_replicas/configs/remote_servers.xml'] to /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/configs/config.d Setup database dir /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/database Setup logs dir /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Setup directory for instance: node3 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_ddl_worker_replicas/configs/remote_servers.xml'] to /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/configs/config.d Setup database dir /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/database Setup logs dir /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Setup directory for instance: node4 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_ddl_worker_replicas/configs/remote_servers.xml'] to /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/configs/config.d Setup database dir /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/database Setup logs dir /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/logs Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon -- ; coproc tail -f /dev/null; wait $$!" Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'LSAN_OPTIONS': 'suppressions=/etc/clickhouse-server/lsan_suppressions.txt', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:5ccda723c1fc', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/.env Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/.env --project-name roottestddlworkerreplicas-gw1 --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/docker-compose.yml pull] Stderr: zoo3 Skipped - Image is already being pulled by zoo2 Stderr: zoo1 Skipped - Image is already being pulled by zoo2 Stderr: node2 Skipped - Image is already being pulled by zoo2 Stderr: node3 Skipped - Image is already being pulled by zoo2 Stderr: node4 Skipped - Image is already being pulled by zoo2 Stderr: node1 Skipped - Image is already being pulled by zoo2 Stderr: zoo2 Pulling Stderr: zoo2 Pulled Setup ZooKeeper Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper1/log', '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper1/config', '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper1/coordination', '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper2/log', '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper2/config', '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper2/coordination', '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper3/log', '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper3/config', '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper3/coordination'] Command:[docker compose --project-name roottestddlworkerreplicas-gw1 --env-file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --verbose up -d] Stderr:time="2026-04-10T18:17:09Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Network roottestddlworkerreplicas-gw1_default Creating Stderr: Network roottestddlworkerreplicas-gw1_default Created Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Creating Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Creating Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Creating Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Created Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Created Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Created Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Starting Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Starting Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Starting Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Started Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Started Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Started Stderr:time="2026-04-10T18:17:11Z" level=debug msg="otel error" error="" Stderr:time="2026-04-10T18:17:11Z" level=debug msg="otel error" error="" Wait ZooKeeper to start get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.1.2, port:2181, use_ssl:False Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.1.3, port:2181, use_ssl:False Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.1.4, port:2181, use_ssl:False Connecting to 172.16.1.4(172.16.1.4):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/.env --project-name roottestddlworkerreplicas-gw1 --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/.env --project-name roottestddlworkerreplicas-gw1 --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/docker-compose.yml up -d --no-recreate] Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Running Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Running Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Running Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Creating Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Creating Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Creating Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Creating Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Created Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Created Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Created Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Created Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Starting Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Starting Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Starting Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Starting Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Started Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Started Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Started Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Started ClickHouse instance created get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node1-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node1-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node1, ip: 172.16.1.8... http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node1-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None ClickHouse node1 started get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node2-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node2-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node2, ip: 172.16.1.7... http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node2-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/b6b3a3d00aa5da6a67a53b5a1a304648738aa99e0df8f982ca69ddefd6baafa5/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/b6b3a3d00aa5da6a67a53b5a1a304648738aa99e0df8f982ca69ddefd6baafa5/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/b6b3a3d00aa5da6a67a53b5a1a304648738aa99e0df8f982ca69ddefd6baafa5/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/b6b3a3d00aa5da6a67a53b5a1a304648738aa99e0df8f982ca69ddefd6baafa5/json HTTP/1.1" 200 None ClickHouse node2 started get_instance_ip instance_name=node3 http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node3-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node3 http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node3-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node3, ip: 172.16.1.5... http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node3-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/8c1f1724dff79f173695fd9c0afaa7bd086b0fea51e126f05f2362c44675a377/json HTTP/1.1" 200 None ClickHouse node3 started get_instance_ip instance_name=node4 http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node4-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node4 http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node4-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node4, ip: 172.16.1.6... http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node4-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/bf424c39e4440194bbe2f64e66a40d67aa6a425a9d7f0da70f697ce23a2fc7ea/json HTTP/1.1" 200 None ClickHouse node4 started ------------------------------ Captured log setup ------------------------------ 2026-04-10 18:17:07.204000 [ 606 ] INFO : Running tests in /ClickHouse/tests/integration/test_ddl_worker_replicas/test.py (cluster.py:2748, start) 2026-04-10 18:17:07.205000 [ 606 ] DEBUG : Cluster start called. is_up=False (cluster.py:2755, start) 2026-04-10 18:17:07.239000 [ 606 ] DEBUG : Docker networks for project roottestddlworkerreplicas-gw1 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 18:17:07.270000 [ 606 ] DEBUG : Docker containers for project roottestddlworkerreplicas-gw1 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 18:17:07.307000 [ 606 ] DEBUG : Docker volumes for project roottestddlworkerreplicas-gw1 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 18:17:07.308000 [ 606 ] DEBUG : Cleanup called (cluster.py:861, cleanup) 2026-04-10 18:17:07.357000 [ 606 ] DEBUG : Docker networks for project roottestddlworkerreplicas-gw1 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 18:17:07.387000 [ 606 ] DEBUG : Docker containers for project roottestddlworkerreplicas-gw1 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 18:17:07.424000 [ 606 ] DEBUG : Docker volumes for project roottestddlworkerreplicas-gw1 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 18:17:07.425000 [ 606 ] DEBUG : Command:[docker container list --all --filter name='^/roottestddlworkerreplicas-gw1-.*-1$' --format '{{.ID}}:{{.Names}}'] (cluster.py:121, run_and_check) 2026-04-10 18:17:07.458000 [ 606 ] DEBUG : Unstopped containers: {} (cluster.py:875, cleanup) 2026-04-10 18:17:07.459000 [ 606 ] DEBUG : No running containers for project: roottestddlworkerreplicas-gw1 (cluster.py:889, cleanup) 2026-04-10 18:17:07.459000 [ 606 ] DEBUG : Trying to prune unused networks... (cluster.py:895, cleanup) 2026-04-10 18:17:07.488000 [ 606 ] DEBUG : Trying to prune unused images... (cluster.py:911, cleanup) 2026-04-10 18:17:07.488000 [ 606 ] DEBUG : Command:[docker image prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:17:07.526000 [ 606 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 18:17:07.527000 [ 606 ] DEBUG : Images pruned (cluster.py:914, cleanup) 2026-04-10 18:17:07.527000 [ 606 ] DEBUG : Trying to prune unused volumes... (cluster.py:920, cleanup) 2026-04-10 18:17:07.527000 [ 606 ] DEBUG : Command:[docker volume ls | wc -l] (cluster.py:121, run_and_check) 2026-04-10 18:17:07.560000 [ 606 ] DEBUG : Stdout:9 (cluster.py:145, run_and_check) 2026-04-10 18:17:07.560000 [ 606 ] DEBUG : Command:[docker volume prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:17:07.604000 [ 606 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 18:17:07.605000 [ 606 ] DEBUG : Volumes pruned: 9 (cluster.py:925, cleanup) 2026-04-10 18:17:07.605000 [ 606 ] DEBUG : Setup directory for instance: node1 (cluster.py:2768, start) 2026-04-10 18:17:07.606000 [ 606 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 18:17:07.606000 [ 606 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 18:17:07.607000 [ 606 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 18:17:07.608000 [ 606 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 18:17:07.609000 [ 606 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_ddl_worker_replicas/configs/remote_servers.xml'] to /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 18:17:07.610000 [ 606 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/database (cluster.py:4772, create_dir) 2026-04-10 18:17:07.610000 [ 606 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/logs (cluster.py:4783, create_dir) 2026-04-10 18:17:07.611000 [ 606 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon -- ; coproc tail -f /dev/null; wait $$!" (cluster.py:4864, create_dir) 2026-04-10 18:17:07.611000 [ 606 ] DEBUG : Setup directory for instance: node2 (cluster.py:2768, start) 2026-04-10 18:17:07.612000 [ 606 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 18:17:07.612000 [ 606 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 18:17:07.612000 [ 606 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 18:17:07.614000 [ 606 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 18:17:07.614000 [ 606 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_ddl_worker_replicas/configs/remote_servers.xml'] to /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 18:17:07.615000 [ 606 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/database (cluster.py:4772, create_dir) 2026-04-10 18:17:07.615000 [ 606 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/logs (cluster.py:4783, create_dir) 2026-04-10 18:17:07.615000 [ 606 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4864, create_dir) 2026-04-10 18:17:07.615000 [ 606 ] DEBUG : Setup directory for instance: node3 (cluster.py:2768, start) 2026-04-10 18:17:07.616000 [ 606 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 18:17:07.616000 [ 606 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 18:17:07.616000 [ 606 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 18:17:07.617000 [ 606 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 18:17:07.618000 [ 606 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_ddl_worker_replicas/configs/remote_servers.xml'] to /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 18:17:07.619000 [ 606 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/database (cluster.py:4772, create_dir) 2026-04-10 18:17:07.619000 [ 606 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/logs (cluster.py:4783, create_dir) 2026-04-10 18:17:07.619000 [ 606 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4864, create_dir) 2026-04-10 18:17:07.619000 [ 606 ] DEBUG : Setup directory for instance: node4 (cluster.py:2768, start) 2026-04-10 18:17:07.620000 [ 606 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 18:17:07.620000 [ 606 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 18:17:07.620000 [ 606 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 18:17:07.621000 [ 606 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 18:17:07.622000 [ 606 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_ddl_worker_replicas/configs/remote_servers.xml'] to /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 18:17:07.622000 [ 606 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/database (cluster.py:4772, create_dir) 2026-04-10 18:17:07.622000 [ 606 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/logs (cluster.py:4783, create_dir) 2026-04-10 18:17:07.623000 [ 606 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon -- ; coproc tail -f /dev/null; wait $$!" (cluster.py:4864, create_dir) 2026-04-10 18:17:07.623000 [ 606 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'LSAN_OPTIONS': 'suppressions=/etc/clickhouse-server/lsan_suppressions.txt', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:5ccda723c1fc', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/.env (cluster.py:96, _create_env_file) 2026-04-10 18:17:07.623000 [ 606 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-04-10 18:17:07.624000 [ 606 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-04-10 18:17:07.624000 [ 606 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-04-10 18:17:07.624000 [ 606 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-04-10 18:17:07.649000 [ 606 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 826 (connectionpool.py:547, _make_request) 2026-04-10 18:17:07.651000 [ 606 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/.env --project-name roottestddlworkerreplicas-gw1 --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/docker-compose.yml pull] (cluster.py:121, run_and_check) 2026-04-10 18:17:09.705000 [ 606 ] DEBUG : Stderr: zoo3 Skipped - Image is already being pulled by zoo2 (cluster.py:147, run_and_check) 2026-04-10 18:17:09.706000 [ 606 ] DEBUG : Stderr: zoo1 Skipped - Image is already being pulled by zoo2 (cluster.py:147, run_and_check) 2026-04-10 18:17:09.706000 [ 606 ] DEBUG : Stderr: node2 Skipped - Image is already being pulled by zoo2 (cluster.py:147, run_and_check) 2026-04-10 18:17:09.706000 [ 606 ] DEBUG : Stderr: node3 Skipped - Image is already being pulled by zoo2 (cluster.py:147, run_and_check) 2026-04-10 18:17:09.706000 [ 606 ] DEBUG : Stderr: node4 Skipped - Image is already being pulled by zoo2 (cluster.py:147, run_and_check) 2026-04-10 18:17:09.706000 [ 606 ] DEBUG : Stderr: node1 Skipped - Image is already being pulled by zoo2 (cluster.py:147, run_and_check) 2026-04-10 18:17:09.706000 [ 606 ] DEBUG : Stderr: zoo2 Pulling (cluster.py:147, run_and_check) 2026-04-10 18:17:09.706000 [ 606 ] DEBUG : Stderr: zoo2 Pulled (cluster.py:147, run_and_check) 2026-04-10 18:17:09.707000 [ 606 ] DEBUG : Setup ZooKeeper (cluster.py:2809, start) 2026-04-10 18:17:09.707000 [ 606 ] DEBUG : Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper1/log', '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper1/config', '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper1/coordination', '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper2/log', '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper2/config', '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper2/coordination', '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper3/log', '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper3/config', '/ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/keeper3/coordination'] (cluster.py:2810, start) 2026-04-10 18:17:09.708000 [ 606 ] DEBUG : Command:[docker compose --project-name roottestddlworkerreplicas-gw1 --env-file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --verbose up -d] (cluster.py:121, run_and_check) 2026-04-10 18:17:11.443000 [ 606 ] DEBUG : Stderr:time="2026-04-10T18:17:09Z" level=trace msg="Docker Desktop integration not enabled" (cluster.py:147, run_and_check) 2026-04-10 18:17:11.443000 [ 606 ] DEBUG : Stderr: Network roottestddlworkerreplicas-gw1_default Creating (cluster.py:147, run_and_check) 2026-04-10 18:17:11.444000 [ 606 ] DEBUG : Stderr: Network roottestddlworkerreplicas-gw1_default Created (cluster.py:147, run_and_check) 2026-04-10 18:17:11.444000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:17:11.444000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:17:11.444000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:17:11.444000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:17:11.444000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:17:11.444000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:17:11.444000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:17:11.444000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:17:11.444000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:17:11.445000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:17:11.445000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:17:11.445000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:17:11.445000 [ 606 ] DEBUG : Stderr:time="2026-04-10T18:17:11Z" level=debug msg="otel error" error="" (cluster.py:147, run_and_check) 2026-04-10 18:17:11.445000 [ 606 ] DEBUG : Stderr:time="2026-04-10T18:17:11Z" level=debug msg="otel error" error="" (cluster.py:147, run_and_check) 2026-04-10 18:17:11.445000 [ 606 ] DEBUG : Wait ZooKeeper to start (cluster.py:2446, wait_zookeeper_to_start) 2026-04-10 18:17:11.445000 [ 606 ] DEBUG : get_instance_ip instance_name=zoo1 (cluster.py:2015, get_instance_ip) 2026-04-10 18:17:11.458000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-zoo1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:11.460000 [ 606 ] DEBUG : get_kazoo_client: zoo1, ip:172.16.1.2, port:2181, use_ssl:False (cluster.py:3322, get_kazoo_client) 2026-04-10 18:17:11.472000 [ 606 ] INFO : Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 18:17:11.473000 [ 606 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-04-10 18:17:11.571000 [ 606 ] INFO : Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 18:17:11.579000 [ 606 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-04-10 18:17:11.784000 [ 606 ] INFO : Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 18:17:11.784000 [ 606 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-04-10 18:17:12.296000 [ 606 ] INFO : Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 18:17:12.299000 [ 606 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-04-10 18:17:13.354000 [ 606 ] INFO : Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 18:17:13.355000 [ 606 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-04-10 18:17:14.705000 [ 606 ] INFO : Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 18:17:14.706000 [ 606 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-04-10 18:17:14.735000 [ 606 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-04-10 18:17:14.738000 [ 606 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-04-10 18:17:14.743000 [ 606 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-04-10 18:17:14.744000 [ 606 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-04-10 18:17:14.752000 [ 606 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-04-10 18:17:14.753000 [ 606 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-04-10 18:17:14.753000 [ 606 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-04-10 18:17:14.855000 [ 606 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-04-10 18:17:14.856000 [ 606 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-04-10 18:17:14.858000 [ 606 ] DEBUG : get_instance_ip instance_name=zoo2 (cluster.py:2015, get_instance_ip) 2026-04-10 18:17:14.862000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-zoo2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:14.863000 [ 606 ] DEBUG : get_kazoo_client: zoo2, ip:172.16.1.3, port:2181, use_ssl:False (cluster.py:3322, get_kazoo_client) 2026-04-10 18:17:14.871000 [ 606 ] INFO : Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 18:17:14.875000 [ 606 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-04-10 18:17:14.885000 [ 606 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-04-10 18:17:14.887000 [ 606 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-04-10 18:17:14.888000 [ 606 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-04-10 18:17:14.889000 [ 606 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-04-10 18:17:14.898000 [ 606 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-04-10 18:17:14.899000 [ 606 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-04-10 18:17:14.899000 [ 606 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-04-10 18:17:14.986000 [ 606 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-04-10 18:17:14.986000 [ 606 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-04-10 18:17:14.987000 [ 606 ] DEBUG : get_instance_ip instance_name=zoo3 (cluster.py:2015, get_instance_ip) 2026-04-10 18:17:14.991000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-zoo3-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:14.992000 [ 606 ] DEBUG : get_kazoo_client: zoo3, ip:172.16.1.4, port:2181, use_ssl:False (cluster.py:3322, get_kazoo_client) 2026-04-10 18:17:14.995000 [ 606 ] INFO : Connecting to 172.16.1.4(172.16.1.4):2181, use_ssl: False (connection.py:650, _connect) 2026-04-10 18:17:14.999000 [ 606 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-04-10 18:17:15.019000 [ 606 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-04-10 18:17:15.023000 [ 606 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-04-10 18:17:15.031000 [ 606 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-04-10 18:17:15.039000 [ 606 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-04-10 18:17:15.050000 [ 606 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-04-10 18:17:15.051000 [ 606 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-04-10 18:17:15.052000 [ 606 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-04-10 18:17:15.152000 [ 606 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-04-10 18:17:15.153000 [ 606 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-04-10 18:17:15.153000 [ 606 ] DEBUG : All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') (cluster.py:2462, wait_zookeeper_nodes_to_start) 2026-04-10 18:17:15.154000 [ 606 ] DEBUG : ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/.env --project-name roottestddlworkerreplicas-gw1 --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/docker-compose.yml up -d --no-recreate') (cluster.py:3149, start) 2026-04-10 18:17:15.154000 [ 606 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/.env --project-name roottestddlworkerreplicas-gw1 --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/docker-compose.yml up -d --no-recreate] (cluster.py:121, run_and_check) 2026-04-10 18:17:16.378000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Running (cluster.py:147, run_and_check) 2026-04-10 18:17:16.378000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Running (cluster.py:147, run_and_check) 2026-04-10 18:17:16.378000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Running (cluster.py:147, run_and_check) 2026-04-10 18:17:16.379000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:17:16.379000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:17:16.379000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:17:16.379000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:17:16.379000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:17:16.379000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:17:16.379000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:17:16.380000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:17:16.380000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:17:16.380000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:17:16.380000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:17:16.380000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:17:16.380000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:17:16.380000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:17:16.380000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:17:16.380000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:17:16.380000 [ 606 ] DEBUG : ClickHouse instance created (cluster.py:3157, start) 2026-04-10 18:17:16.380000 [ 606 ] DEBUG : get_instance_ip instance_name=node1 (cluster.py:2015, get_instance_ip) 2026-04-10 18:17:16.385000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:16.386000 [ 606 ] DEBUG : get_instance_ip instance_name=node1 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:17:17.031000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:17.033000 [ 606 ] DEBUG : Waiting for ClickHouse start in node1, ip: 172.16.1.8... (cluster.py:3165, start) 2026-04-10 18:17:17.036000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:17.053000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:17.167000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:17.279000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:17.398000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:17.515000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:17.621000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:17.725000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:17.831000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:17.938000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:18.049000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:18.158000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:18.266000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:18.371000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:18.476000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:18.581000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:18.686000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:18.793000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:18.899000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:19.017000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:19.128000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:19.234000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:19.346000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:19.455000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:19.564000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:19.676000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:19.789000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:19.896000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:20.002000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:20.112000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:20.230000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:20.339000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:20.445000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:20.551000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/18b37d6aecf43bc5c54ce1b3089dec6e9446f1ef3afba335660ee4dc6562f3db/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:20.552000 [ 606 ] DEBUG : ClickHouse node1 started (cluster.py:3169, start) 2026-04-10 18:17:20.553000 [ 606 ] DEBUG : get_instance_ip instance_name=node2 (cluster.py:2015, get_instance_ip) 2026-04-10 18:17:20.559000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:20.560000 [ 606 ] DEBUG : get_instance_ip instance_name=node2 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:17:20.565000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:20.566000 [ 606 ] DEBUG : Waiting for ClickHouse start in node2, ip: 172.16.1.7... (cluster.py:3165, start) 2026-04-10 18:17:20.573000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:20.584000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/b6b3a3d00aa5da6a67a53b5a1a304648738aa99e0df8f982ca69ddefd6baafa5/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:20.735000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/b6b3a3d00aa5da6a67a53b5a1a304648738aa99e0df8f982ca69ddefd6baafa5/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:20.847000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/b6b3a3d00aa5da6a67a53b5a1a304648738aa99e0df8f982ca69ddefd6baafa5/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:20.976000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/b6b3a3d00aa5da6a67a53b5a1a304648738aa99e0df8f982ca69ddefd6baafa5/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:20.977000 [ 606 ] DEBUG : ClickHouse node2 started (cluster.py:3169, start) 2026-04-10 18:17:20.978000 [ 606 ] DEBUG : get_instance_ip instance_name=node3 (cluster.py:2015, get_instance_ip) 2026-04-10 18:17:20.981000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node3-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:20.982000 [ 606 ] DEBUG : get_instance_ip instance_name=node3 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:17:20.985000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node3-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:20.985000 [ 606 ] DEBUG : Waiting for ClickHouse start in node3, ip: 172.16.1.5... (cluster.py:3165, start) 2026-04-10 18:17:20.988000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node3-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:20.993000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/8c1f1724dff79f173695fd9c0afaa7bd086b0fea51e126f05f2362c44675a377/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:20.994000 [ 606 ] DEBUG : ClickHouse node3 started (cluster.py:3169, start) 2026-04-10 18:17:20.994000 [ 606 ] DEBUG : get_instance_ip instance_name=node4 (cluster.py:2015, get_instance_ip) 2026-04-10 18:17:21.000000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node4-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:21.002000 [ 606 ] DEBUG : get_instance_ip instance_name=node4 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:17:21.004000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node4-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:21.006000 [ 606 ] DEBUG : Waiting for ClickHouse start in node4, ip: 172.16.1.6... (cluster.py:3165, start) 2026-04-10 18:17:21.012000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestddlworkerreplicas-gw1-node4-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:21.017000 [ 606 ] DEBUG : http://localhost:None "GET /v1.46/containers/bf424c39e4440194bbe2f64e66a40d67aa6a425a9d7f0da70f697ce23a2fc7ea/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:17:21.018000 [ 606 ] DEBUG : ClickHouse node4 started (cluster.py:3169, start) ----------------------------- Captured stdout call ----------------------------- result: node1:9000 ----------------------------- Captured stderr call ----------------------------- Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 Executing query SELECT name, value, ephemeralOwner FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 ------------------------------ Captured log call ------------------------------- 2026-04-10 18:17:21.023000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:22.516000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:23.861000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:24.845000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:25.823000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:26.794000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:27.721000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:28.709000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:30.300000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:34.703000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:35.835000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:41.731000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:43.118000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:44.154000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:50.970000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:52.223000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:53.332000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:54.359000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:55.230000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:56.108000 [ 606 ] DEBUG : Executing query SELECT count() FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) 2026-04-10 18:17:57.338000 [ 606 ] DEBUG : Executing query SELECT name, value, ephemeralOwner FROM system.zookeeper WHERE path='/clickhouse/task_queue/replicas/node1:9000' on node1 (cluster.py:3658, query) --------------------------- Captured stderr teardown --------------------------- Command:[docker compose --env-file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/.env --project-name roottestddlworkerreplicas-gw1 --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/docker-compose.yml stop --timeout 20] Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Stopping Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Stopping Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Stopping Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Stopping Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Stopped Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Stopped Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Stopped Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Stopped Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Stopping Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Stopping Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Stopping Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Stopped Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Stopped Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/.env --project-name roottestddlworkerreplicas-gw1 --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/docker-compose.yml down --volumes] Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Stopping Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Stopping Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Stopping Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Stopping Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Stopped Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Removing Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Stopped Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Removing Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Stopped Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Removing Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Stopped Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Removing Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Removed Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Removed Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Removed Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Removed Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Stopping Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Stopping Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Stopping Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Stopped Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Removing Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Stopped Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Removing Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Stopped Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Removing Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Removed Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Removed Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Removed Stderr: Network roottestddlworkerreplicas-gw1_default Removing Stderr: Network roottestddlworkerreplicas-gw1_default Removed Cleanup called Docker networks for project roottestddlworkerreplicas-gw1 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestddlworkerreplicas-gw1 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestddlworkerreplicas-gw1 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestddlworkerreplicas-gw1-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestddlworkerreplicas-gw1 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:7 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 7 ---------------------------- Captured log teardown ----------------------------- 2026-04-10 18:17:57.933000 [ 606 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/.env --project-name roottestddlworkerreplicas-gw1 --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/docker-compose.yml stop --timeout 20] (cluster.py:121, run_and_check) 2026-04-10 18:18:04.291000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:18:04.291000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:18:04.291000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:18:04.292000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:18:04.292000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:18:04.292000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:18:04.292000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:18:04.292000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:18:04.292000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:18:04.292000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:18:04.292000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:18:04.293000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:18:04.293000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:18:04.293000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:18:04.293000 [ 606 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:18:04.322000 [ 606 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:18:04.346000 [ 606 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:18:04.366000 [ 606 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:18:04.386000 [ 606 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/.env --project-name roottestddlworkerreplicas-gw1 --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_ddl_worker_replicas/_instances-0-gw1/node4/docker-compose.yml down --volumes] (cluster.py:121, run_and_check) 2026-04-10 18:18:05.281000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:18:05.281000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:18:05.281000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:18:05.282000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:18:05.282000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:18:05.282000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:18:05.282000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:18:05.282000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:18:05.282000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:18:05.282000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:18:05.283000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:18:05.283000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:18:05.283000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node2-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:18:05.283000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node1-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:18:05.283000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node3-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:18:05.283000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-node4-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:18:05.283000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:18:05.283000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:18:05.284000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:18:05.284000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:18:05.284000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:18:05.284000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:18:05.284000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:18:05.284000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:18:05.284000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:18:05.284000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo1-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:18:05.285000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo3-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:18:05.285000 [ 606 ] DEBUG : Stderr: Container roottestddlworkerreplicas-gw1-zoo2-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:18:05.285000 [ 606 ] DEBUG : Stderr: Network roottestddlworkerreplicas-gw1_default Removing (cluster.py:147, run_and_check) 2026-04-10 18:18:05.285000 [ 606 ] DEBUG : Stderr: Network roottestddlworkerreplicas-gw1_default Removed (cluster.py:147, run_and_check) 2026-04-10 18:18:05.286000 [ 606 ] DEBUG : Cleanup called (cluster.py:861, cleanup) 2026-04-10 18:18:05.317000 [ 606 ] DEBUG : Docker networks for project roottestddlworkerreplicas-gw1 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 18:18:05.347000 [ 606 ] DEBUG : Docker containers for project roottestddlworkerreplicas-gw1 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 18:18:05.380000 [ 606 ] DEBUG : Docker volumes for project roottestddlworkerreplicas-gw1 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 18:18:05.381000 [ 606 ] DEBUG : Command:[docker container list --all --filter name='^/roottestddlworkerreplicas-gw1-.*-1$' --format '{{.ID}}:{{.Names}}'] (cluster.py:121, run_and_check) 2026-04-10 18:18:05.421000 [ 606 ] DEBUG : Unstopped containers: {} (cluster.py:875, cleanup) 2026-04-10 18:18:05.422000 [ 606 ] DEBUG : No running containers for project: roottestddlworkerreplicas-gw1 (cluster.py:889, cleanup) 2026-04-10 18:18:05.422000 [ 606 ] DEBUG : Trying to prune unused networks... (cluster.py:895, cleanup) 2026-04-10 18:18:05.461000 [ 606 ] DEBUG : Trying to prune unused images... (cluster.py:911, cleanup) 2026-04-10 18:18:05.462000 [ 606 ] DEBUG : Command:[docker image prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:18:05.504000 [ 606 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 18:18:05.505000 [ 606 ] DEBUG : Images pruned (cluster.py:914, cleanup) 2026-04-10 18:18:05.505000 [ 606 ] DEBUG : Trying to prune unused volumes... (cluster.py:920, cleanup) 2026-04-10 18:18:05.505000 [ 606 ] DEBUG : Command:[docker volume ls | wc -l] (cluster.py:121, run_and_check) 2026-04-10 18:18:05.534000 [ 606 ] DEBUG : Stdout:7 (cluster.py:145, run_and_check) 2026-04-10 18:18:05.535000 [ 606 ] DEBUG : Command:[docker volume prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:18:05.571000 [ 606 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 18:18:05.571000 [ 606 ] DEBUG : Volumes pruned: 7 (cluster.py:925, cleanup) _____________________________ test_both_mergetree ______________________________ [gw5] linux -- Python 3.10.12 /usr/bin/python3 start_cluster = def test_both_mergetree(start_cluster): cleanup([replica1, replica2]) > create_source_table(replica1, "source", False) test_attach_partition_using_copy/test.py:106: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ test_attach_partition_using_copy/test.py:40: in create_source_table node.query_with_retry( _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = sql = "\n ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7'\n (\n price UInt32,\n ...disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/')\n " stdin = None, timeout = 60, settings = None, user = None, password = None database = None, host = None, ignore_error = False, retry_count = 3 sleep_time = 0.5 check_callback = at 0x7fb883bb5630> parse = False def query_with_retry( self, sql, stdin=None, timeout=None, settings=None, user=None, password=None, database=None, host=None, ignore_error=False, retry_count=20, sleep_time=0.5, check_callback=lambda x: True, parse=False, ): # logging.debug(f"Executing query {sql} on {self.name}") result = None exception_msg = "" for i in range(retry_count): try: result = self.query( sql, stdin=stdin, timeout=timeout, settings=settings, user=user, password=password, database=database, host=host, ignore_error=ignore_error, parse=parse, ) if check_callback(result): return result time.sleep(sleep_time) except QueryRuntimeException as ex: exception_msg = f"{type(ex).__name__}: {str(ex)}" # Container is down, this is likely due to server crash. if "No route to host" in str(ex): raise time.sleep(sleep_time) except Exception as ex: # logging.debug("Retry {} got exception {}".format(i + 1, ex)) exception_msg = f"{type(ex).__name__}: {str(ex)}" time.sleep(sleep_time) if result is not None: return result > raise Exception(f"Can't execute query {sql}\n{exception_msg}") E Exception: Can't execute query E ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' E ( E price UInt32, E date Date, E postcode1 LowCardinality(String), E postcode2 LowCardinality(String), E type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), E is_new UInt8, E duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), E addr1 String, E addr2 String, E street LowCardinality(String), E locality LowCardinality(String), E town LowCardinality(String), E district LowCardinality(String), E county LowCardinality(String) E ) E ENGINE = MergeTree() E ORDER BY (postcode1, postcode2, addr1, addr2) E SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') E E QueryRuntimeException: Client failed! Return code: 198, stderr: Received exception from server (version 25.3.8): E Code: 198. DB::Exception: Received from 172.16.2.5:9000. DB::NetException. DB::NetException: Not found address of host: raw.githubusercontent.com: while loading disk metadata. Stack trace: E E 0. ./contrib/llvm-project/libcxx/include/__exception/exception.h:113: Poco::Exception::Exception(String const&, int) @ 0x0000000036f39bb1 E 1. ./build_docker/./src/Common/Exception.cpp:108: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001b1bca51 E 2. ./src/Common/Exception.h:112: DB::NetException::NetException(int, FormatStringHelperImpl::type>, String const&) @ 0x000000001b14702f E 3. ./build_docker/./src/Common/DNSResolver.cpp:113: DB::(anonymous namespace)::hostByName(String const&) @ 0x000000001b1401de E 4. ./build_docker/./src/Common/DNSResolver.cpp:138: DB::DNSResolver::getResolvedIPAdressessWithFiltering(String const&) @ 0x000000001b13d168 E 5. ./build_docker/./src/Common/DNSResolver.cpp:256: DB::DNSResolver::resolveIPAddressWithCache(String const&) @ 0x000000001b13dba0 E 6. ./build_docker/./src/Common/DNSResolver.cpp:276: DB::DNSResolver::resolveHostAllInOriginOrder(String const&) @ 0x000000001b13e8e7 E 7. ./build_docker/./src/Common/HostResolvePool.cpp:54: std::vector> std::__function::__policy_invoker> (String const&)>::__call_impl[abi:ne190107]> (String const&)>>(std::__function::__policy_storage const*, String const&) @ 0x000000001b9595a0 E 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001b955d21 E 9. ./build_docker/./src/Common/HostResolvePool.cpp:66: DB::HostResolver::HostResolver(std::function> (String const&)>&&, String, Poco::Timespan) @ 0x000000001b9557ed E 10. ./build_docker/./src/Common/HostResolvePool.cpp:53: DB::HostResolver::HostResolver(String, Poco::Timespan) @ 0x000000001b9551b4 E 11. ./src/Common/HostResolvePool.h:62: std::shared_ptr DB::HostResolver::create(String const&)::make_shared_enabler::make_shared_enabler(String const&) @ 0x000000001b95ea10 E 12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:41: std::shared_ptr DB::HostResolver::create(String const&)::make_shared_enabler> std::allocate_shared[abi:ne190107] DB::HostResolver::create(String const&)::make_shared_enabler, std::allocator DB::HostResolver::create(String const&)::make_shared_enabler>, String const&, 0>(std::allocator DB::HostResolver::create(String const&)::make_shared_enabler> const&, String const&) @ 0x000000001b95e58d E 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:851: DB::HostResolversPool::getResolver(String const&) @ 0x000000001b95938b E 14. ./build_docker/./src/Common/HTTPConnectionPool.cpp:671: DB::EndpointConnectionPool::prepareNewConnection(DB::ConnectionTimeouts const&, unsigned long*) @ 0x000000001b940b28 E 15. ./build_docker/./src/Common/HTTPConnectionPool.cpp:590: DB::EndpointConnectionPool::getConnection(DB::ConnectionTimeouts const&, unsigned long*) @ 0x000000001b93eeb0 E 16. ./build_docker/./src/IO/HTTPCommon.cpp:63: DB::makeHTTPSession(DB::HTTPConnectionGroupType, Poco::URI const&, DB::ConnectionTimeouts const&, DB::ProxyConfiguration const&, unsigned long*) @ 0x000000001b969334 E 17. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:267: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x00000000206707b2 E 18. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:285: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x0000000020670e3c E 19. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:408: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000020671ebb E 20. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:472: void std::__function::__policy_invoker::__call_impl[abi:ne190107]>(std::__function::__policy_storage const*) @ 0x00000000206777d8 E 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002066c831 E 22. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:465: DB::ReadWriteBufferFromHTTP::nextImpl() @ 0x00000000206744e3 E 23. DB::ReadBuffer::next() @ 0x000000000ba328eb E 24. ./src/IO/ReadBuffer.h:96: DB::WebObjectStorage::loadFiles(String const&, std::unique_lock const&) const @ 0x0000000027a861c2 E 25. ./build_docker/./src/Disks/ObjectStorages/Web/WebObjectStorage.cpp:225: DB::WebObjectStorage::tryGetFileInfo(String const&) const @ 0x0000000027a8ab1f E 26. ./build_docker/./src/Disks/ObjectStorages/Web/WebObjectStorage.cpp:185: DB::WebObjectStorage::tryGetFileInfo(String const&) const @ 0x0000000027a8a450 E 27. ./build_docker/./src/Disks/ObjectStorages/Web/MetadataStorageFromStaticFilesWebServer.cpp:106: DB::MetadataStorageFromStaticFilesWebServer::getStorageObjectsIfExist(String const&) const @ 0x0000000027a82586 E 28. ./build_docker/./src/Disks/ObjectStorages/DiskObjectStorage.cpp:785: DB::DiskObjectStorage::readFileIfExists(String const&, DB::ReadSettings const&, std::optional, std::optional) const @ 0x000000002798e67e E 29. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:380: DB::MergeTreeData::initializeDirectoriesAndFormatVersion(String const&, bool, String const&, bool) @ 0x000000002e338f90 E 30. ./build_docker/./src/Storages/StorageMergeTree.cpp:159: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr>) @ 0x000000002eaf0ff6 E 31. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:41: std::shared_ptr std::allocate_shared[abi:ne190107], DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>, 0>(std::allocator const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>&&) @ 0x000000002eaf0656 E . (DNS_ERROR) E (query: ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' E ( E price UInt32, E date Date, E postcode1 LowCardinality(String), E postcode2 LowCardinality(String), E type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), E is_new UInt8, E duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), E addr1 String, E addr2 String, E street LowCardinality(String), E locality LowCardinality(String), E town LowCardinality(String), E district LowCardinality(String), E county LowCardinality(String) E ) E ENGINE = MergeTree() E ORDER BY (postcode1, postcode2, addr1, addr2) E SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') E ) helpers/cluster.py:3722: Exception ----------------------------- Captured stderr call ----------------------------- Executing query DROP TABLE IF EXISTS source SYNC on replica1 Executing query DROP TABLE IF EXISTS destination SYNC on replica1 Executing query DROP TABLE IF EXISTS source SYNC on replica2 Executing query DROP TABLE IF EXISTS destination SYNC on replica2 Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree() ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree() ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree() ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 ------------------------------ Captured log call ------------------------------- 2026-04-10 18:16:54.740000 [ 618 ] DEBUG : Executing query DROP TABLE IF EXISTS source SYNC on replica1 (cluster.py:3658, query) 2026-04-10 18:16:55.059000 [ 618 ] DEBUG : Executing query DROP TABLE IF EXISTS destination SYNC on replica1 (cluster.py:3658, query) 2026-04-10 18:16:55.378000 [ 618 ] DEBUG : Executing query DROP TABLE IF EXISTS source SYNC on replica2 (cluster.py:3658, query) 2026-04-10 18:16:55.698000 [ 618 ] DEBUG : Executing query DROP TABLE IF EXISTS destination SYNC on replica2 (cluster.py:3658, query) 2026-04-10 18:16:56.122000 [ 618 ] DEBUG : Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree() ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 (cluster.py:3658, query) 2026-04-10 18:17:55.731000 [ 618 ] DEBUG : Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree() ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 (cluster.py:3658, query) 2026-04-10 18:18:50.153000 [ 618 ] DEBUG : Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree() ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 (cluster.py:3658, query) ___________________ test_cow_policy[cow_policy_multi_volume] ___________________ [gw7] linux -- Python 3.10.12 /usr/bin/python3 start_cluster = storage_policy = 'cow_policy_multi_volume' @pytest.mark.parametrize("storage_policy", ["cow_policy_multi_disk", "cow_policy_multi_volume"]) def test_cow_policy(start_cluster, storage_policy): try: > node.query_with_retry( f""" ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS storage_policy = '{storage_policy}' """, timeout=60, retry_count=3, ) test_cow_policy/test.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = sql = "\n ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7'\n (\n ...BY (postcode1, postcode2, addr1, addr2)\n SETTINGS storage_policy = 'cow_policy_multi_volume'\n " stdin = None, timeout = 60, settings = None, user = None, password = None database = None, host = None, ignore_error = False, retry_count = 3 sleep_time = 0.5 check_callback = at 0x7f5b38871630> parse = False def query_with_retry( self, sql, stdin=None, timeout=None, settings=None, user=None, password=None, database=None, host=None, ignore_error=False, retry_count=20, sleep_time=0.5, check_callback=lambda x: True, parse=False, ): # logging.debug(f"Executing query {sql} on {self.name}") result = None exception_msg = "" for i in range(retry_count): try: result = self.query( sql, stdin=stdin, timeout=timeout, settings=settings, user=user, password=password, database=database, host=host, ignore_error=ignore_error, parse=parse, ) if check_callback(result): return result time.sleep(sleep_time) except QueryRuntimeException as ex: exception_msg = f"{type(ex).__name__}: {str(ex)}" # Container is down, this is likely due to server crash. if "No route to host" in str(ex): raise time.sleep(sleep_time) except Exception as ex: # logging.debug("Retry {} got exception {}".format(i + 1, ex)) exception_msg = f"{type(ex).__name__}: {str(ex)}" time.sleep(sleep_time) if result is not None: return result > raise Exception(f"Can't execute query {sql}\n{exception_msg}") E Exception: Can't execute query E ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' E ( E price UInt32, E date Date, E postcode1 LowCardinality(String), E postcode2 LowCardinality(String), E type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), E is_new UInt8, E duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), E addr1 String, E addr2 String, E street LowCardinality(String), E locality LowCardinality(String), E town LowCardinality(String), E district LowCardinality(String), E county LowCardinality(String) E ) E ENGINE = MergeTree E ORDER BY (postcode1, postcode2, addr1, addr2) E SETTINGS storage_policy = 'cow_policy_multi_volume' E E QueryRuntimeException: Client failed! Return code: 198, stderr: Received exception from server (version 25.3.8): E Code: 198. DB::Exception: Received from 172.16.9.2:9000. DB::NetException. DB::NetException: Not found address of host: raw.githubusercontent.com: while loading disk metadata. Stack trace: E E 0. ./contrib/llvm-project/libcxx/include/__exception/exception.h:113: Poco::Exception::Exception(String const&, int) @ 0x0000000036f39bb1 E 1. ./build_docker/./src/Common/Exception.cpp:108: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001b1bca51 E 2. ./src/Common/Exception.h:112: DB::NetException::NetException(int, FormatStringHelperImpl::type>, String const&) @ 0x000000001b14702f E 3. ./build_docker/./src/Common/DNSResolver.cpp:113: DB::(anonymous namespace)::hostByName(String const&) @ 0x000000001b1401de E 4. ./build_docker/./src/Common/DNSResolver.cpp:138: DB::DNSResolver::getResolvedIPAdressessWithFiltering(String const&) @ 0x000000001b13d168 E 5. ./build_docker/./src/Common/DNSResolver.cpp:256: DB::DNSResolver::resolveIPAddressWithCache(String const&) @ 0x000000001b13dba0 E 6. ./build_docker/./src/Common/DNSResolver.cpp:276: DB::DNSResolver::resolveHostAllInOriginOrder(String const&) @ 0x000000001b13e8e7 E 7. ./build_docker/./src/Common/HostResolvePool.cpp:54: std::vector> std::__function::__policy_invoker> (String const&)>::__call_impl[abi:ne190107]> (String const&)>>(std::__function::__policy_storage const*, String const&) @ 0x000000001b9595a0 E 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001b955d21 E 9. ./build_docker/./src/Common/HostResolvePool.cpp:66: DB::HostResolver::HostResolver(std::function> (String const&)>&&, String, Poco::Timespan) @ 0x000000001b9557ed E 10. ./build_docker/./src/Common/HostResolvePool.cpp:53: DB::HostResolver::HostResolver(String, Poco::Timespan) @ 0x000000001b9551b4 E 11. ./src/Common/HostResolvePool.h:62: std::shared_ptr DB::HostResolver::create(String const&)::make_shared_enabler::make_shared_enabler(String const&) @ 0x000000001b95ea10 E 12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:41: std::shared_ptr DB::HostResolver::create(String const&)::make_shared_enabler> std::allocate_shared[abi:ne190107] DB::HostResolver::create(String const&)::make_shared_enabler, std::allocator DB::HostResolver::create(String const&)::make_shared_enabler>, String const&, 0>(std::allocator DB::HostResolver::create(String const&)::make_shared_enabler> const&, String const&) @ 0x000000001b95e58d E 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:851: DB::HostResolversPool::getResolver(String const&) @ 0x000000001b95938b E 14. ./build_docker/./src/Common/HTTPConnectionPool.cpp:671: DB::EndpointConnectionPool::prepareNewConnection(DB::ConnectionTimeouts const&, unsigned long*) @ 0x000000001b940b28 E 15. ./build_docker/./src/Common/HTTPConnectionPool.cpp:590: DB::EndpointConnectionPool::getConnection(DB::ConnectionTimeouts const&, unsigned long*) @ 0x000000001b93eeb0 E 16. ./build_docker/./src/IO/HTTPCommon.cpp:63: DB::makeHTTPSession(DB::HTTPConnectionGroupType, Poco::URI const&, DB::ConnectionTimeouts const&, DB::ProxyConfiguration const&, unsigned long*) @ 0x000000001b969334 E 17. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:267: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x00000000206707b2 E 18. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:285: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x0000000020670e3c E 19. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:408: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000020671ebb E 20. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:472: void std::__function::__policy_invoker::__call_impl[abi:ne190107]>(std::__function::__policy_storage const*) @ 0x00000000206777d8 E 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002066c831 E 22. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:465: DB::ReadWriteBufferFromHTTP::nextImpl() @ 0x00000000206744e3 E 23. DB::ReadBuffer::next() @ 0x000000000ba328eb E 24. ./src/IO/ReadBuffer.h:96: DB::WebObjectStorage::loadFiles(String const&, std::unique_lock const&) const @ 0x0000000027a861c2 E 25. ./build_docker/./src/Disks/ObjectStorages/Web/WebObjectStorage.cpp:225: DB::WebObjectStorage::tryGetFileInfo(String const&) const @ 0x0000000027a8ab1f E 26. ./build_docker/./src/Disks/ObjectStorages/Web/WebObjectStorage.cpp:185: DB::WebObjectStorage::tryGetFileInfo(String const&) const @ 0x0000000027a8a450 E 27. ./build_docker/./src/Disks/ObjectStorages/Web/MetadataStorageFromStaticFilesWebServer.cpp:106: DB::MetadataStorageFromStaticFilesWebServer::getStorageObjectsIfExist(String const&) const @ 0x0000000027a82586 E 28. ./build_docker/./src/Disks/ObjectStorages/DiskObjectStorage.cpp:785: DB::DiskObjectStorage::readFileIfExists(String const&, DB::ReadSettings const&, std::optional, std::optional) const @ 0x000000002798e67e E 29. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:380: DB::MergeTreeData::initializeDirectoriesAndFormatVersion(String const&, bool, String const&, bool) @ 0x000000002e338f90 E 30. ./build_docker/./src/Storages/StorageMergeTree.cpp:159: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr>) @ 0x000000002eaf0ff6 E 31. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:41: std::shared_ptr std::allocate_shared[abi:ne190107], DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>, 0>(std::allocator const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>&&) @ 0x000000002eaf0656 E . (DNS_ERROR) E (query: ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' E ( E price UInt32, E date Date, E postcode1 LowCardinality(String), E postcode2 LowCardinality(String), E type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), E is_new UInt8, E duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), E addr1 String, E addr2 String, E street LowCardinality(String), E locality LowCardinality(String), E town LowCardinality(String), E district LowCardinality(String), E county LowCardinality(String) E ) E ENGINE = MergeTree E ORDER BY (postcode1, postcode2, addr1, addr2) E SETTINGS storage_policy = 'cow_policy_multi_volume' E ) helpers/cluster.py:3722: Exception ----------------------------- Captured stderr call ----------------------------- Executing query ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS storage_policy = 'cow_policy_multi_volume' on node Timed out. Last stdout:, stderr: Executing query ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS storage_policy = 'cow_policy_multi_volume' on node Timed out. Last stdout:, stderr: Executing query ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS storage_policy = 'cow_policy_multi_volume' on node Executing query DROP TABLE IF EXISTS uk_price_paid SYNC on node ------------------------------ Captured log call ------------------------------- 2026-04-10 18:17:36.251000 [ 624 ] DEBUG : Executing query ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS storage_policy = 'cow_policy_multi_volume' on node (cluster.py:3658, query) 2026-04-10 18:18:36.315000 [ 624 ] DEBUG : Timed out. Last stdout:, stderr: (client.py:242, get_answer) 2026-04-10 18:18:36.817000 [ 624 ] DEBUG : Executing query ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS storage_policy = 'cow_policy_multi_volume' on node (cluster.py:3658, query) 2026-04-10 18:19:36.851000 [ 624 ] DEBUG : Timed out. Last stdout:, stderr: (client.py:242, get_answer) 2026-04-10 18:19:37.352000 [ 624 ] DEBUG : Executing query ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS storage_policy = 'cow_policy_multi_volume' on node (cluster.py:3658, query) 2026-04-10 18:20:33.800000 [ 624 ] DEBUG : Executing query DROP TABLE IF EXISTS uk_price_paid SYNC on node (cluster.py:3658, query) --------------------------- Captured stderr teardown --------------------------- Command:[docker compose --env-file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/.env --project-name roottestcowpolicy-gw7 --file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/docker-compose.yml stop --timeout 20] Stderr: Container roottestcowpolicy-gw7-node-1 Stopping Stderr: Container roottestcowpolicy-gw7-node-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/.env --project-name roottestcowpolicy-gw7 --file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/docker-compose.yml down --volumes] Stderr: Container roottestcowpolicy-gw7-node-1 Stopping Stderr: Container roottestcowpolicy-gw7-node-1 Stopped Stderr: Container roottestcowpolicy-gw7-node-1 Removing Stderr: Container roottestcowpolicy-gw7-node-1 Removed Stderr: Network roottestcowpolicy-gw7_default Removing Stderr: Network roottestcowpolicy-gw7_default Removed Cleanup called Docker networks for project roottestcowpolicy-gw7 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestcowpolicy-gw7 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestcowpolicy-gw7 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestcowpolicy-gw7-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestcowpolicy-gw7 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:7 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 7 ---------------------------- Captured log teardown ----------------------------- 2026-04-10 18:20:34.188000 [ 624 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/.env --project-name roottestcowpolicy-gw7 --file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/docker-compose.yml stop --timeout 20] (cluster.py:121, run_and_check) 2026-04-10 18:20:40.228000 [ 624 ] DEBUG : Stderr: Container roottestcowpolicy-gw7-node-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:20:40.229000 [ 624 ] DEBUG : Stderr: Container roottestcowpolicy-gw7-node-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:20:40.229000 [ 624 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:20:40.240000 [ 624 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/.env --project-name roottestcowpolicy-gw7 --file /ClickHouse/tests/integration/test_cow_policy/_instances-0-gw7/node/docker-compose.yml down --volumes] (cluster.py:121, run_and_check) 2026-04-10 18:20:40.833000 [ 624 ] DEBUG : Stderr: Container roottestcowpolicy-gw7-node-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:20:40.833000 [ 624 ] DEBUG : Stderr: Container roottestcowpolicy-gw7-node-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:20:40.834000 [ 624 ] DEBUG : Stderr: Container roottestcowpolicy-gw7-node-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:20:40.834000 [ 624 ] DEBUG : Stderr: Container roottestcowpolicy-gw7-node-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:20:40.834000 [ 624 ] DEBUG : Stderr: Network roottestcowpolicy-gw7_default Removing (cluster.py:147, run_and_check) 2026-04-10 18:20:40.834000 [ 624 ] DEBUG : Stderr: Network roottestcowpolicy-gw7_default Removed (cluster.py:147, run_and_check) 2026-04-10 18:20:40.834000 [ 624 ] DEBUG : Cleanup called (cluster.py:861, cleanup) 2026-04-10 18:20:40.860000 [ 624 ] DEBUG : Docker networks for project roottestcowpolicy-gw7 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 18:20:40.883000 [ 624 ] DEBUG : Docker containers for project roottestcowpolicy-gw7 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 18:20:40.905000 [ 624 ] DEBUG : Docker volumes for project roottestcowpolicy-gw7 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 18:20:40.906000 [ 624 ] DEBUG : Command:[docker container list --all --filter name='^/roottestcowpolicy-gw7-.*-1$' --format '{{.ID}}:{{.Names}}'] (cluster.py:121, run_and_check) 2026-04-10 18:20:40.927000 [ 624 ] DEBUG : Unstopped containers: {} (cluster.py:875, cleanup) 2026-04-10 18:20:40.928000 [ 624 ] DEBUG : No running containers for project: roottestcowpolicy-gw7 (cluster.py:889, cleanup) 2026-04-10 18:20:40.928000 [ 624 ] DEBUG : Trying to prune unused networks... (cluster.py:895, cleanup) 2026-04-10 18:20:40.952000 [ 624 ] DEBUG : Trying to prune unused images... (cluster.py:911, cleanup) 2026-04-10 18:20:40.952000 [ 624 ] DEBUG : Command:[docker image prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:20:40.984000 [ 624 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 18:20:40.984000 [ 624 ] DEBUG : Images pruned (cluster.py:914, cleanup) 2026-04-10 18:20:40.985000 [ 624 ] DEBUG : Trying to prune unused volumes... (cluster.py:920, cleanup) 2026-04-10 18:20:40.985000 [ 624 ] DEBUG : Command:[docker volume ls | wc -l] (cluster.py:121, run_and_check) 2026-04-10 18:20:41.010000 [ 624 ] DEBUG : Stdout:7 (cluster.py:145, run_and_check) 2026-04-10 18:20:41.010000 [ 624 ] DEBUG : Command:[docker volume prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:20:41.032000 [ 624 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 18:20:41.032000 [ 624 ] DEBUG : Volumes pruned: 7 (cluster.py:925, cleanup) _______________________ test_not_work_on_different_disk ________________________ [gw5] linux -- Python 3.10.12 /usr/bin/python3 start_cluster = def test_not_work_on_different_disk(start_cluster): cleanup([replica1, replica2]) # Replace and move should not work on replace > create_source_table(replica1, "source", False) test_attach_partition_using_copy/test.py:199: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ test_attach_partition_using_copy/test.py:40: in create_source_table node.query_with_retry( _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = sql = "\n ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7'\n (\n price UInt32,\n ...disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/')\n " stdin = None, timeout = 60, settings = None, user = None, password = None database = None, host = None, ignore_error = False, retry_count = 3 sleep_time = 0.5 check_callback = at 0x7fb883bb5630> parse = False def query_with_retry( self, sql, stdin=None, timeout=None, settings=None, user=None, password=None, database=None, host=None, ignore_error=False, retry_count=20, sleep_time=0.5, check_callback=lambda x: True, parse=False, ): # logging.debug(f"Executing query {sql} on {self.name}") result = None exception_msg = "" for i in range(retry_count): try: result = self.query( sql, stdin=stdin, timeout=timeout, settings=settings, user=user, password=password, database=database, host=host, ignore_error=ignore_error, parse=parse, ) if check_callback(result): return result time.sleep(sleep_time) except QueryRuntimeException as ex: exception_msg = f"{type(ex).__name__}: {str(ex)}" # Container is down, this is likely due to server crash. if "No route to host" in str(ex): raise time.sleep(sleep_time) except Exception as ex: # logging.debug("Retry {} got exception {}".format(i + 1, ex)) exception_msg = f"{type(ex).__name__}: {str(ex)}" time.sleep(sleep_time) if result is not None: return result > raise Exception(f"Can't execute query {sql}\n{exception_msg}") E Exception: Can't execute query E ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' E ( E price UInt32, E date Date, E postcode1 LowCardinality(String), E postcode2 LowCardinality(String), E type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), E is_new UInt8, E duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), E addr1 String, E addr2 String, E street LowCardinality(String), E locality LowCardinality(String), E town LowCardinality(String), E district LowCardinality(String), E county LowCardinality(String) E ) E ENGINE = MergeTree() E ORDER BY (postcode1, postcode2, addr1, addr2) E SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') E E QueryRuntimeException: Client failed! Return code: 198, stderr: Received exception from server (version 25.3.8): E Code: 198. DB::Exception: Received from 172.16.2.5:9000. DB::NetException. DB::NetException: Not found address of host: raw.githubusercontent.com: while loading disk metadata. Stack trace: E E 0. ./contrib/llvm-project/libcxx/include/__exception/exception.h:113: Poco::Exception::Exception(String const&, int) @ 0x0000000036f39bb1 E 1. ./build_docker/./src/Common/Exception.cpp:108: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001b1bca51 E 2. ./src/Common/Exception.h:112: DB::NetException::NetException(int, FormatStringHelperImpl::type>, String const&) @ 0x000000001b14702f E 3. ./build_docker/./src/Common/DNSResolver.cpp:113: DB::(anonymous namespace)::hostByName(String const&) @ 0x000000001b1401de E 4. ./build_docker/./src/Common/DNSResolver.cpp:138: DB::DNSResolver::getResolvedIPAdressessWithFiltering(String const&) @ 0x000000001b13d168 E 5. ./build_docker/./src/Common/DNSResolver.cpp:256: DB::DNSResolver::resolveIPAddressWithCache(String const&) @ 0x000000001b13dba0 E 6. ./build_docker/./src/Common/DNSResolver.cpp:276: DB::DNSResolver::resolveHostAllInOriginOrder(String const&) @ 0x000000001b13e8e7 E 7. ./build_docker/./src/Common/HostResolvePool.cpp:54: std::vector> std::__function::__policy_invoker> (String const&)>::__call_impl[abi:ne190107]> (String const&)>>(std::__function::__policy_storage const*, String const&) @ 0x000000001b9595a0 E 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001b955d21 E 9. ./build_docker/./src/Common/HostResolvePool.cpp:66: DB::HostResolver::HostResolver(std::function> (String const&)>&&, String, Poco::Timespan) @ 0x000000001b9557ed E 10. ./build_docker/./src/Common/HostResolvePool.cpp:53: DB::HostResolver::HostResolver(String, Poco::Timespan) @ 0x000000001b9551b4 E 11. ./src/Common/HostResolvePool.h:62: std::shared_ptr DB::HostResolver::create(String const&)::make_shared_enabler::make_shared_enabler(String const&) @ 0x000000001b95ea10 E 12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:41: std::shared_ptr DB::HostResolver::create(String const&)::make_shared_enabler> std::allocate_shared[abi:ne190107] DB::HostResolver::create(String const&)::make_shared_enabler, std::allocator DB::HostResolver::create(String const&)::make_shared_enabler>, String const&, 0>(std::allocator DB::HostResolver::create(String const&)::make_shared_enabler> const&, String const&) @ 0x000000001b95e58d E 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:851: DB::HostResolversPool::getResolver(String const&) @ 0x000000001b95938b E 14. ./build_docker/./src/Common/HTTPConnectionPool.cpp:671: DB::EndpointConnectionPool::prepareNewConnection(DB::ConnectionTimeouts const&, unsigned long*) @ 0x000000001b940b28 E 15. ./build_docker/./src/Common/HTTPConnectionPool.cpp:590: DB::EndpointConnectionPool::getConnection(DB::ConnectionTimeouts const&, unsigned long*) @ 0x000000001b93eeb0 E 16. ./build_docker/./src/IO/HTTPCommon.cpp:63: DB::makeHTTPSession(DB::HTTPConnectionGroupType, Poco::URI const&, DB::ConnectionTimeouts const&, DB::ProxyConfiguration const&, unsigned long*) @ 0x000000001b969334 E 17. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:267: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x00000000206707b2 E 18. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:285: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x0000000020670e3c E 19. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:408: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000020671ebb E 20. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:472: void std::__function::__policy_invoker::__call_impl[abi:ne190107]>(std::__function::__policy_storage const*) @ 0x00000000206777d8 E 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002066c831 E 22. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:465: DB::ReadWriteBufferFromHTTP::nextImpl() @ 0x00000000206744e3 E 23. DB::ReadBuffer::next() @ 0x000000000ba328eb E 24. ./src/IO/ReadBuffer.h:96: DB::WebObjectStorage::loadFiles(String const&, std::unique_lock const&) const @ 0x0000000027a861c2 E 25. ./build_docker/./src/Disks/ObjectStorages/Web/WebObjectStorage.cpp:225: DB::WebObjectStorage::tryGetFileInfo(String const&) const @ 0x0000000027a8ab1f E 26. ./build_docker/./src/Disks/ObjectStorages/Web/WebObjectStorage.cpp:185: DB::WebObjectStorage::tryGetFileInfo(String const&) const @ 0x0000000027a8a450 E 27. ./build_docker/./src/Disks/ObjectStorages/Web/MetadataStorageFromStaticFilesWebServer.cpp:106: DB::MetadataStorageFromStaticFilesWebServer::getStorageObjectsIfExist(String const&) const @ 0x0000000027a82586 E 28. ./build_docker/./src/Disks/ObjectStorages/DiskObjectStorage.cpp:785: DB::DiskObjectStorage::readFileIfExists(String const&, DB::ReadSettings const&, std::optional, std::optional) const @ 0x000000002798e67e E 29. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:380: DB::MergeTreeData::initializeDirectoriesAndFormatVersion(String const&, bool, String const&, bool) @ 0x000000002e338f90 E 30. ./build_docker/./src/Storages/StorageMergeTree.cpp:159: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr>) @ 0x000000002eaf0ff6 E 31. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:41: std::shared_ptr std::allocate_shared[abi:ne190107], DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>, 0>(std::allocator const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>&&) @ 0x000000002eaf0656 E . (DNS_ERROR) E (query: ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' E ( E price UInt32, E date Date, E postcode1 LowCardinality(String), E postcode2 LowCardinality(String), E type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), E is_new UInt8, E duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), E addr1 String, E addr2 String, E street LowCardinality(String), E locality LowCardinality(String), E town LowCardinality(String), E district LowCardinality(String), E county LowCardinality(String) E ) E ENGINE = MergeTree() E ORDER BY (postcode1, postcode2, addr1, addr2) E SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') E ) helpers/cluster.py:3722: Exception ----------------------------- Captured stderr call ----------------------------- Executing query DROP TABLE IF EXISTS source SYNC on replica1 Executing query DROP TABLE IF EXISTS destination SYNC on replica1 Executing query DROP TABLE IF EXISTS source SYNC on replica2 Executing query DROP TABLE IF EXISTS destination SYNC on replica2 Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree() ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree() ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree() ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 ------------------------------ Captured log call ------------------------------- 2026-04-10 18:19:45.581000 [ 618 ] DEBUG : Executing query DROP TABLE IF EXISTS source SYNC on replica1 (cluster.py:3658, query) 2026-04-10 18:19:45.848000 [ 618 ] DEBUG : Executing query DROP TABLE IF EXISTS destination SYNC on replica1 (cluster.py:3658, query) 2026-04-10 18:19:46.165000 [ 618 ] DEBUG : Executing query DROP TABLE IF EXISTS source SYNC on replica2 (cluster.py:3658, query) 2026-04-10 18:19:46.483000 [ 618 ] DEBUG : Executing query DROP TABLE IF EXISTS destination SYNC on replica2 (cluster.py:3658, query) 2026-04-10 18:19:46.801000 [ 618 ] DEBUG : Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree() ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 (cluster.py:3658, query) 2026-04-10 18:20:43.037000 [ 618 ] DEBUG : Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree() ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 (cluster.py:3658, query) 2026-04-10 18:21:40.859000 [ 618 ] DEBUG : Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree() ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 (cluster.py:3658, query) _______________________ test_only_destination_replicated _______________________ [gw5] linux -- Python 3.10.12 /usr/bin/python3 start_cluster = def test_only_destination_replicated(start_cluster): cleanup([replica1, replica2]) > create_source_table(replica1, "source", False) test_attach_partition_using_copy/test.py:163: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ test_attach_partition_using_copy/test.py:40: in create_source_table node.query_with_retry( _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = sql = "\n ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7'\n (\n price UInt32,\n ...disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/')\n " stdin = None, timeout = 60, settings = None, user = None, password = None database = None, host = None, ignore_error = False, retry_count = 3 sleep_time = 0.5 check_callback = at 0x7fb883bb5630> parse = False def query_with_retry( self, sql, stdin=None, timeout=None, settings=None, user=None, password=None, database=None, host=None, ignore_error=False, retry_count=20, sleep_time=0.5, check_callback=lambda x: True, parse=False, ): # logging.debug(f"Executing query {sql} on {self.name}") result = None exception_msg = "" for i in range(retry_count): try: result = self.query( sql, stdin=stdin, timeout=timeout, settings=settings, user=user, password=password, database=database, host=host, ignore_error=ignore_error, parse=parse, ) if check_callback(result): return result time.sleep(sleep_time) except QueryRuntimeException as ex: exception_msg = f"{type(ex).__name__}: {str(ex)}" # Container is down, this is likely due to server crash. if "No route to host" in str(ex): raise time.sleep(sleep_time) except Exception as ex: # logging.debug("Retry {} got exception {}".format(i + 1, ex)) exception_msg = f"{type(ex).__name__}: {str(ex)}" time.sleep(sleep_time) if result is not None: return result > raise Exception(f"Can't execute query {sql}\n{exception_msg}") E Exception: Can't execute query E ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' E ( E price UInt32, E date Date, E postcode1 LowCardinality(String), E postcode2 LowCardinality(String), E type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), E is_new UInt8, E duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), E addr1 String, E addr2 String, E street LowCardinality(String), E locality LowCardinality(String), E town LowCardinality(String), E district LowCardinality(String), E county LowCardinality(String) E ) E ENGINE = MergeTree() E ORDER BY (postcode1, postcode2, addr1, addr2) E SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') E E QueryRuntimeException: Client failed! Return code: 198, stderr: Received exception from server (version 25.3.8): E Code: 198. DB::Exception: Received from 172.16.2.5:9000. DB::NetException. DB::NetException: Not found address of host: raw.githubusercontent.com: while loading disk metadata. Stack trace: E E 0. ./contrib/llvm-project/libcxx/include/__exception/exception.h:113: Poco::Exception::Exception(String const&, int) @ 0x0000000036f39bb1 E 1. ./build_docker/./src/Common/Exception.cpp:108: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001b1bca51 E 2. ./src/Common/Exception.h:112: DB::NetException::NetException(int, FormatStringHelperImpl::type>, String const&) @ 0x000000001b14702f E 3. ./build_docker/./src/Common/DNSResolver.cpp:113: DB::(anonymous namespace)::hostByName(String const&) @ 0x000000001b1401de E 4. ./build_docker/./src/Common/DNSResolver.cpp:138: DB::DNSResolver::getResolvedIPAdressessWithFiltering(String const&) @ 0x000000001b13d168 E 5. ./build_docker/./src/Common/DNSResolver.cpp:256: DB::DNSResolver::resolveIPAddressWithCache(String const&) @ 0x000000001b13dba0 E 6. ./build_docker/./src/Common/DNSResolver.cpp:276: DB::DNSResolver::resolveHostAllInOriginOrder(String const&) @ 0x000000001b13e8e7 E 7. ./build_docker/./src/Common/HostResolvePool.cpp:54: std::vector> std::__function::__policy_invoker> (String const&)>::__call_impl[abi:ne190107]> (String const&)>>(std::__function::__policy_storage const*, String const&) @ 0x000000001b9595a0 E 8. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000001b955d21 E 9. ./build_docker/./src/Common/HostResolvePool.cpp:66: DB::HostResolver::HostResolver(std::function> (String const&)>&&, String, Poco::Timespan) @ 0x000000001b9557ed E 10. ./build_docker/./src/Common/HostResolvePool.cpp:53: DB::HostResolver::HostResolver(String, Poco::Timespan) @ 0x000000001b9551b4 E 11. ./src/Common/HostResolvePool.h:62: std::shared_ptr DB::HostResolver::create(String const&)::make_shared_enabler::make_shared_enabler(String const&) @ 0x000000001b95ea10 E 12. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:41: std::shared_ptr DB::HostResolver::create(String const&)::make_shared_enabler> std::allocate_shared[abi:ne190107] DB::HostResolver::create(String const&)::make_shared_enabler, std::allocator DB::HostResolver::create(String const&)::make_shared_enabler>, String const&, 0>(std::allocator DB::HostResolver::create(String const&)::make_shared_enabler> const&, String const&) @ 0x000000001b95e58d E 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:851: DB::HostResolversPool::getResolver(String const&) @ 0x000000001b95938b E 14. ./build_docker/./src/Common/HTTPConnectionPool.cpp:671: DB::EndpointConnectionPool::prepareNewConnection(DB::ConnectionTimeouts const&, unsigned long*) @ 0x000000001b940b28 E 15. ./build_docker/./src/Common/HTTPConnectionPool.cpp:590: DB::EndpointConnectionPool::getConnection(DB::ConnectionTimeouts const&, unsigned long*) @ 0x000000001b93eeb0 E 16. ./build_docker/./src/IO/HTTPCommon.cpp:63: DB::makeHTTPSession(DB::HTTPConnectionGroupType, Poco::URI const&, DB::ConnectionTimeouts const&, DB::ProxyConfiguration const&, unsigned long*) @ 0x000000001b969334 E 17. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:267: DB::ReadWriteBufferFromHTTP::callImpl(Poco::Net::HTTPResponse&, String const&, std::optional const&, bool) const @ 0x00000000206707b2 E 18. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:285: DB::ReadWriteBufferFromHTTP::callWithRedirects(Poco::Net::HTTPResponse&, String const&, std::optional const&) @ 0x0000000020670e3c E 19. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:408: DB::ReadWriteBufferFromHTTP::initialize() @ 0x0000000020671ebb E 20. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:472: void std::__function::__policy_invoker::__call_impl[abi:ne190107]>(std::__function::__policy_storage const*) @ 0x00000000206777d8 E 21. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x000000002066c831 E 22. ./build_docker/./src/IO/ReadWriteBufferFromHTTP.cpp:465: DB::ReadWriteBufferFromHTTP::nextImpl() @ 0x00000000206744e3 E 23. DB::ReadBuffer::next() @ 0x000000000ba328eb E 24. ./src/IO/ReadBuffer.h:96: DB::WebObjectStorage::loadFiles(String const&, std::unique_lock const&) const @ 0x0000000027a861c2 E 25. ./build_docker/./src/Disks/ObjectStorages/Web/WebObjectStorage.cpp:225: DB::WebObjectStorage::tryGetFileInfo(String const&) const @ 0x0000000027a8ab1f E 26. ./build_docker/./src/Disks/ObjectStorages/Web/WebObjectStorage.cpp:185: DB::WebObjectStorage::tryGetFileInfo(String const&) const @ 0x0000000027a8a450 E 27. ./build_docker/./src/Disks/ObjectStorages/Web/MetadataStorageFromStaticFilesWebServer.cpp:106: DB::MetadataStorageFromStaticFilesWebServer::getStorageObjectsIfExist(String const&) const @ 0x0000000027a82586 E 28. ./build_docker/./src/Disks/ObjectStorages/DiskObjectStorage.cpp:785: DB::DiskObjectStorage::readFileIfExists(String const&, DB::ReadSettings const&, std::optional, std::optional) const @ 0x000000002798e67e E 29. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:380: DB::MergeTreeData::initializeDirectoriesAndFormatVersion(String const&, bool, String const&, bool) @ 0x000000002e338f90 E 30. ./build_docker/./src/Storages/StorageMergeTree.cpp:159: DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr>) @ 0x000000002eaf0ff6 E 31. ./contrib/llvm-project/libcxx/include/__memory/construct_at.h:41: std::shared_ptr std::allocate_shared[abi:ne190107], DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>, 0>(std::allocator const&, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata&, DB::LoadingStrictnessLevel const&, std::shared_ptr&, String&, DB::MergeTreeData::MergingParams&, std::unique_ptr>&&) @ 0x000000002eaf0656 E . (DNS_ERROR) E (query: ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' E ( E price UInt32, E date Date, E postcode1 LowCardinality(String), E postcode2 LowCardinality(String), E type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), E is_new UInt8, E duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), E addr1 String, E addr2 String, E street LowCardinality(String), E locality LowCardinality(String), E town LowCardinality(String), E district LowCardinality(String), E county LowCardinality(String) E ) E ENGINE = MergeTree() E ORDER BY (postcode1, postcode2, addr1, addr2) E SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') E ) helpers/cluster.py:3722: Exception ----------------------------- Captured stderr call ----------------------------- Executing query DROP TABLE IF EXISTS source SYNC on replica1 Executing query DROP TABLE IF EXISTS destination SYNC on replica1 Executing query DROP TABLE IF EXISTS source SYNC on replica2 Executing query DROP TABLE IF EXISTS destination SYNC on replica2 Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree() ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree() ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree() ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 ------------------------------ Captured log call ------------------------------- 2026-04-10 18:22:35.993000 [ 618 ] DEBUG : Executing query DROP TABLE IF EXISTS source SYNC on replica1 (cluster.py:3658, query) 2026-04-10 18:22:36.310000 [ 618 ] DEBUG : Executing query DROP TABLE IF EXISTS destination SYNC on replica1 (cluster.py:3658, query) 2026-04-10 18:22:36.627000 [ 618 ] DEBUG : Executing query DROP TABLE IF EXISTS source SYNC on replica2 (cluster.py:3658, query) 2026-04-10 18:22:36.894000 [ 618 ] DEBUG : Executing query DROP TABLE IF EXISTS destination SYNC on replica2 (cluster.py:3658, query) 2026-04-10 18:22:37.161000 [ 618 ] DEBUG : Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree() ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 (cluster.py:3658, query) 2026-04-10 18:23:31.923000 [ 618 ] DEBUG : Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree() ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 (cluster.py:3658, query) 2026-04-10 18:24:28.997000 [ 618 ] DEBUG : Executing query ATTACH TABLE source UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' ( price UInt32, date Date, postcode1 LowCardinality(String), postcode2 LowCardinality(String), type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), is_new UInt8, duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), addr1 String, addr2 String, street LowCardinality(String), locality LowCardinality(String), town LowCardinality(String), district LowCardinality(String), county LowCardinality(String) ) ENGINE = MergeTree() ORDER BY (postcode1, postcode2, addr1, addr2) SETTINGS disk = disk(type = web, endpoint = 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/') on replica1 (cluster.py:3658, query) --------------------------- Captured stderr teardown --------------------------- Command:[docker compose --env-file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/.env --project-name roottestattachpartitionusingcopy-gw5 --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/docker-compose.yml stop --timeout 20] Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Stopping Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Stopping Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Stopped Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Stopped Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Stopping Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Stopping Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Stopping Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Stopped Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Stopped Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/.env --project-name roottestattachpartitionusingcopy-gw5 --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/docker-compose.yml down --volumes] Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Stopping Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Stopping Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Stopped Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Removing Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Stopped Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Removing Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Removed Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Removed Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Stopping Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Stopping Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Stopping Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Stopped Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Removing Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Stopped Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Removing Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Stopped Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Removing Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Removed Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Removed Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Removed Stderr: Network roottestattachpartitionusingcopy-gw5_default Removing Stderr: Network roottestattachpartitionusingcopy-gw5_default Removed Cleanup called Docker networks for project roottestattachpartitionusingcopy-gw5 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestattachpartitionusingcopy-gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestattachpartitionusingcopy-gw5 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestattachpartitionusingcopy-gw5-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestattachpartitionusingcopy-gw5 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:1 Volumes pruned: 1 ---------------------------- Captured log teardown ----------------------------- 2026-04-10 18:25:26.921000 [ 618 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/.env --project-name roottestattachpartitionusingcopy-gw5 --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/docker-compose.yml stop --timeout 20] (cluster.py:121, run_and_check) 2026-04-10 18:25:34.361000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:25:34.361000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:25:34.361000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:25:34.361000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:25:34.361000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:25:34.362000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:25:34.362000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:25:34.362000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:25:34.362000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:25:34.362000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:25:34.362000 [ 618 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:25:34.375000 [ 618 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:25:34.386000 [ 618 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/.env --project-name roottestattachpartitionusingcopy-gw5 --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_attach_partition_using_copy/_instances-0-gw5/replica2/docker-compose.yml down --volumes] (cluster.py:121, run_and_check) 2026-04-10 18:25:34.886000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:25:34.886000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:25:34.886000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:25:34.886000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:25:34.887000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:25:34.887000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:25:34.887000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica2-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:25:34.887000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-replica1-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:25:34.887000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:25:34.887000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:25:34.887000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:25:34.887000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:25:34.887000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:25:34.888000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:25:34.888000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:25:34.888000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:25:34.888000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:25:34.888000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo2-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:25:34.888000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo1-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:25:34.888000 [ 618 ] DEBUG : Stderr: Container roottestattachpartitionusingcopy-gw5-zoo3-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:25:34.888000 [ 618 ] DEBUG : Stderr: Network roottestattachpartitionusingcopy-gw5_default Removing (cluster.py:147, run_and_check) 2026-04-10 18:25:34.888000 [ 618 ] DEBUG : Stderr: Network roottestattachpartitionusingcopy-gw5_default Removed (cluster.py:147, run_and_check) 2026-04-10 18:25:34.889000 [ 618 ] DEBUG : Cleanup called (cluster.py:861, cleanup) 2026-04-10 18:25:34.911000 [ 618 ] DEBUG : Docker networks for project roottestattachpartitionusingcopy-gw5 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 18:25:34.932000 [ 618 ] DEBUG : Docker containers for project roottestattachpartitionusingcopy-gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 18:25:34.954000 [ 618 ] DEBUG : Docker volumes for project roottestattachpartitionusingcopy-gw5 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 18:25:34.954000 [ 618 ] DEBUG : Command:[docker container list --all --filter name='^/roottestattachpartitionusingcopy-gw5-.*-1$' --format '{{.ID}}:{{.Names}}'] (cluster.py:121, run_and_check) 2026-04-10 18:25:34.976000 [ 618 ] DEBUG : Unstopped containers: {} (cluster.py:875, cleanup) 2026-04-10 18:25:34.976000 [ 618 ] DEBUG : No running containers for project: roottestattachpartitionusingcopy-gw5 (cluster.py:889, cleanup) 2026-04-10 18:25:34.976000 [ 618 ] DEBUG : Trying to prune unused networks... (cluster.py:895, cleanup) 2026-04-10 18:25:34.998000 [ 618 ] DEBUG : Trying to prune unused images... (cluster.py:911, cleanup) 2026-04-10 18:25:34.998000 [ 618 ] DEBUG : Command:[docker image prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:25:35.026000 [ 618 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 18:25:35.026000 [ 618 ] DEBUG : Images pruned (cluster.py:914, cleanup) 2026-04-10 18:25:35.026000 [ 618 ] DEBUG : Trying to prune unused volumes... (cluster.py:920, cleanup) 2026-04-10 18:25:35.027000 [ 618 ] DEBUG : Command:[docker volume ls | wc -l] (cluster.py:121, run_and_check) 2026-04-10 18:25:35.049000 [ 618 ] DEBUG : Stdout:1 (cluster.py:145, run_and_check) 2026-04-10 18:25:35.049000 [ 618 ] DEBUG : Volumes pruned: 1 (cluster.py:925, cleanup) ----------------- generated report log file: parallel0_0.jsonl ----------------- ============================== slowest durations =============================== 193.51s setup test_backup_restore_azure_blob_storage/test.py::test_backup_restore 178.78s call test_cow_policy/test.py::test_cow_policy[cow_policy_multi_disk] 177.82s call test_cow_policy/test.py::test_cow_policy[cow_policy_multi_volume] 170.84s call test_attach_partition_using_copy/test.py::test_only_destination_replicated 170.63s call test_attach_partition_using_copy/test.py::test_both_mergetree 169.90s call test_attach_partition_using_copy/test.py::test_not_work_on_different_disk 167.70s call test_attach_partition_using_copy/test.py::test_all_replicated 143.23s call test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas 128.49s call test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated 92.35s call test_cluster_discovery/test_auxiliary_keeper.py::test_cluster_discovery_with_auxiliary_keeper_startup_and_stop 84.97s setup test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability 82.67s call test_distributed_directory_monitor_split_batch_on_failure/test.py::test_distributed_background_insert_split_batch_on_failure_ON 79.80s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_hashed-False-False] 78.17s call test_distributed_directory_monitor_split_batch_on_failure/test.py::test_distributed_background_insert_split_batch_on_failure_OFF 78.07s setup test_backward_compatibility/test_ip_types_binary_compatibility.py::test_ip_types_binary_compatibility 71.45s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_cache-False-False] 65.04s call test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_simple[hashed] 59.64s call test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_complex[complex_key_hashed] 57.55s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_direct-False-False] 57.29s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[cache-False-False] 55.68s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_cache-True-False] 53.83s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[direct-True-True] 52.71s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[flat-True-True] 51.94s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[direct-False-False] 51.45s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[cache-True-True] 51.31s call test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool 51.31s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[direct-True-False] 50.96s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_direct-True-False] 49.49s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[hashed-True-True] 48.55s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[hashed-False-True] 48.47s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[direct-False-True] 48.08s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[cache-True-False] 47.84s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[flat-False-True] 47.54s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[flat-True-False] 46.73s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_hashed-True-False] 46.72s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_cache-False-False] 46.27s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[flat-False-False] 46.17s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[hashed-False-False] 45.83s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[hashed-True-False] 45.25s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[cache-False-True] 41.48s setup test_cluster_discovery/test_password.py::test_connect_with_password 36.75s call test_ddl_worker_replicas/test.py::test_ddl_worker_replicas 32.84s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[cache-False-True] 32.83s call test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_ranged[range_hashed] 31.14s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_cache-True-False] 30.52s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[cache-True-True] 30.10s teardown test_backup_restore_azure_blob_storage/test.py::test_backup_restore_with_named_collection_azure_conf2 29.35s setup test_config_reloader_interval/test.py::test_reload_config 27.41s setup test_ddl_alter_query/test.py::test_alter 26.13s call test_backward_compatibility/test_ip_types_binary_compatibility.py::test_ip_types_binary_compatibility 25.83s teardown test_ddl_alter_query/test.py::test_ddl_queue_hostname_change 25.64s setup test_distributed_ddl_password/test.py::test_alter 25.47s setup test_file_cluster/test.py::test_count 25.23s setup test_executable_user_defined_function/test.py::test_executable_function_always_error_python 24.96s teardown test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-s3_no_native_copy] 24.14s setup test_cluster_discovery/test_auxiliary_keeper.py::test_cluster_discovery_with_auxiliary_keeper_startup_and_stop 22.80s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_ranged[range_hashed-False-False] 22.45s setup test_check_table_name_length/test.py::test_backward_compatibility 21.90s setup test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool 21.64s call test_config_yaml_merge_keys/test.py::test_yaml_merge_keys_conf 21.52s setup test_compression_nested_columns/test.py::test_nested_compression_codec 20.83s setup test_compatibility_merge_tree_settings/test.py::test_check_projections_compatibility 20.53s call test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_ranged[range_hashed-True-False] 20.35s setup test_attach_partition_using_copy/test.py::test_all_replicated 19.85s setup test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-default] 19.20s call test_distributed_ddl_password/test.py::test_alter 18.68s setup test_asynchronous_metric_log_table/test.py::test_event_time_microseconds_field 17.40s setup test_default_role/test.py::test_alter_user 17.37s setup test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_complex[complex_key_hashed] 16.99s call test_config_xml_main/test.py::test_xml_main_conf 16.94s setup test_concurrent_queries_for_user_restriction/test.py::test_exception_message 16.76s setup test_cleanup_after_start/test.py::test_old_dirs_cleanup 16.34s setup test_buffer_profile/test.py::test_buffer_profile 16.31s setup test_backward_compatibility/test_short_strings_aggregation.py::test_backward_compatability 16.09s teardown test_cluster_discovery/test_password.py::test_connect_with_password 15.57s setup test_accept_invalid_certificate/test.py::test_accept 14.95s setup test_cow_policy/test.py::test_cow_policy[cow_policy_multi_disk] 14.16s setup test_custom_settings/test.py::test_custom_settings 14.06s call test_cleanup_after_start/test.py::test_old_dirs_cleanup 13.82s setup test_ddl_worker_replicas/test.py::test_ddl_worker_replicas 13.71s setup test_distributed_directory_monitor_split_batch_on_failure/test.py::test_distributed_background_insert_split_batch_on_failure_OFF 13.66s call test_executable_user_defined_function/test.py::test_executable_function_slow_python 12.48s call test_ddl_alter_query/test.py::test_ddl_queue_hostname_change 12.36s call test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool 12.16s call test_distributed_ddl_password/test.py::test_truncate 12.03s call test_ddl_alter_query/test.py::test_alter 11.35s teardown test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool 11.03s teardown test_check_table_name_length/test.py::test_check_table_name_length 10.01s teardown test_compatibility_merge_tree_settings/test.py::test_config_overrides_compatibility 9.79s call test_backup_restore_azure_blob_storage/test.py::test_backup_restore_correct_block_ids 9.01s teardown test_file_cluster/test.py::test_format_detection 8.87s teardown test_distributed_ddl_password/test.py::test_truncate 8.25s teardown test_config_reloader_interval/test.py::test_reload_config 8.13s teardown test_attach_partition_using_copy/test.py::test_only_destination_replicated 7.65s teardown test_ddl_worker_replicas/test.py::test_ddl_worker_replicas 7.50s teardown test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_simple[hashed] 7.49s teardown test_compression_nested_columns/test.py::test_nested_compression_codec 7.22s call test_compatibility_merge_tree_settings/test.py::test_check_projections_compatibility 7.19s call test_file_cluster/test.py::test_format_detection 7.18s call test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability 7.16s teardown test_distributed_directory_monitor_split_batch_on_failure/test.py::test_distributed_background_insert_split_batch_on_failure_ON 6.98s call test_executable_user_defined_function/test.py::test_executable_function_sum_json_python 6.96s call test_asynchronous_metric_log_table/test.py::test_event_time_microseconds_field 6.85s teardown test_cow_policy/test.py::test_cow_policy[cow_policy_multi_volume] 6.62s call test_cluster_discovery/test_password.py::test_connect_with_password 6.24s teardown test_buffer_profile/test.py::test_default_profile 6.13s teardown test_asynchronous_metric_log_table/test.py::test_event_time_microseconds_field 6.09s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[hashed-True-True] 5.96s teardown test_backward_compatibility/test_short_strings_aggregation.py::test_backward_compatability 5.89s teardown test_concurrent_queries_for_user_restriction/test.py::test_exception_message 5.84s teardown test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability 5.45s teardown test_accept_invalid_certificate/test.py::test_strict_reject_with_config 5.40s teardown test_default_role/test.py::test_wrong_set_default_role 5.34s call test_default_role/test.py::test_set_default_roles 5.19s teardown test_custom_settings/test.py::test_illformed_setting 4.72s teardown test_cleanup_after_start/test.py::test_old_dirs_cleanup 4.45s call test_backward_compatibility/test_short_strings_aggregation.py::test_backward_compatability 4.39s call test_compression_nested_columns/test.py::test_nested_compression_codec 4.33s call test_executable_user_defined_function/test.py::test_executable_function_send_chunk_header_python 4.27s call test_executable_user_defined_function/test.py::test_executable_function_always_error_python 4.15s call test_default_role/test.py::test_alter_user 4.03s call test_executable_user_defined_function/test.py::test_executable_function_query_cache 3.92s call test_executable_user_defined_function/test.py::test_executable_function_input_nullable_python 3.85s teardown test_cluster_discovery/test_auxiliary_keeper.py::test_cluster_discovery_with_auxiliary_keeper_startup_and_stop 3.65s call test_backup_restore_azure_blob_storage/test.py::test_backup_restore_on_merge_tree 3.30s call test_backup_restore_azure_blob_storage/test.py::test_backup_restore_with_named_collection_azure_conf2 3.24s call test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-s3_no_native_copy] 3.14s call test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-s3_no_native_copy] 3.01s call test_backup_restore_azure_blob_storage/test.py::test_backup_restore_with_named_collection_azure_conf1 2.98s teardown test_backward_compatibility/test_ip_types_binary_compatibility.py::test_ip_types_binary_compatibility 2.84s call test_executable_user_defined_function/test.py::test_executable_function_parameter_python 2.82s teardown test_executable_user_defined_function/test.py::test_executable_function_sum_python 2.79s call test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-default] 2.66s call test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-default] 2.63s call test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-s3_native_copy] 2.50s call test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-s3_native_copy] 2.32s call test_backup_restore_azure_blob_storage/test.py::test_backup_restore 2.31s call test_custom_settings/test.py::test_custom_settings 2.29s call test_executable_user_defined_function/test.py::test_executable_function_argument_python 2.28s call test_backup_restore_azure_blob_storage/test.py::test_backup_restore_diff_container 2.27s call test_executable_user_defined_function/test.py::test_executable_function_python 2.11s call test_default_role/test.py::test_wrong_set_default_role 2.06s call test_executable_user_defined_function/test.py::test_executable_function_bash 2.02s call test_executable_user_defined_function/test.py::test_executable_function_sum_python 1.99s call test_concurrent_queries_for_user_restriction/test.py::test_exception_message 1.96s call test_executable_user_defined_function/test.py::test_executable_function_non_direct_bash 1.87s call test_executable_user_defined_function/test.py::test_executable_function_signalled_python 1.77s call test_config_reloader_interval/test.py::test_reload_config 1.63s call test_check_table_name_length/test.py::test_check_table_name_length 1.50s call test_check_table_name_length/test.py::test_backward_compatibility 1.41s call test_custom_settings/test.py::test_illformed_setting 0.86s call test_buffer_profile/test.py::test_buffer_profile 0.84s call test_file_cluster/test.py::test_count 0.82s call test_compatibility_merge_tree_settings/test.py::test_config_overrides_compatibility 0.81s call test_buffer_profile/test.py::test_default_profile 0.48s call test_accept_invalid_certificate/test.py::test_accept 0.47s call test_accept_invalid_certificate/test.py::test_connection_accept 0.43s setup test_default_role/test.py::test_set_default_roles 0.39s call test_accept_invalid_certificate/test.py::test_default 0.37s call test_accept_invalid_certificate/test.py::test_strict_connection_reject 0.37s call test_accept_invalid_certificate/test.py::test_strict_reject 0.33s setup test_default_role/test.py::test_wrong_set_default_role 0.32s call test_accept_invalid_certificate/test.py::test_strict_reject_with_config 0.01s teardown test_backup_restore_azure_blob_storage/test.py::test_backup_restore 0.01s teardown test_default_role/test.py::test_set_default_roles 0.01s teardown test_backup_restore_azure_blob_storage/test.py::test_backup_restore_diff_container 0.01s teardown test_distributed_ddl_password/test.py::test_alter 0.01s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_hashed-False-False] 0.01s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_cache-False-False] 0.01s teardown test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated 0.01s teardown test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_ranged[range_hashed] 0.00s teardown test_executable_user_defined_function/test.py::test_executable_function_always_error_python 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_ranged[range_hashed-False-False] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_direct-False-False] 0.00s setup test_backup_restore_azure_blob_storage/test.py::test_backup_restore_on_merge_tree 0.00s setup test_distributed_ddl_password/test.py::test_truncate 0.00s setup test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_direct-False-False] 0.00s teardown test_config_yaml_merge_keys/test.py::test_yaml_merge_keys_conf 0.00s setup test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_simple[hashed] 0.00s setup test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas 0.00s teardown test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-s3_native_copy] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[cache-False-False] 0.00s teardown test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas 0.00s setup test_executable_user_defined_function/test.py::test_executable_function_bash 0.00s teardown test_ddl_alter_query/test.py::test_alter 0.00s setup test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool 0.00s setup test_custom_settings/test.py::test_illformed_setting 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_direct-True-False] 0.00s teardown test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-s3_no_native_copy] 0.00s setup test_config_yaml_merge_keys/test.py::test_yaml_merge_keys_conf 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_cache-True-False] 0.00s teardown test_attach_partition_using_copy/test.py::test_both_mergetree 0.00s setup test_config_xml_main/test.py::test_xml_main_conf 0.00s teardown test_attach_partition_using_copy/test.py::test_all_replicated 0.00s setup test_accept_invalid_certificate/test.py::test_strict_reject 0.00s setup test_attach_partition_using_copy/test.py::test_not_work_on_different_disk 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_hashed-False-False] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[hashed-True-True] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[direct-False-False] 0.00s teardown test_cow_policy/test.py::test_cow_policy[cow_policy_multi_disk] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[cache-True-True] 0.00s teardown test_compatibility_merge_tree_settings/test.py::test_check_projections_compatibility 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_complex[complex_key_hashed] 0.00s teardown test_check_table_name_length/test.py::test_backward_compatibility 0.00s teardown test_executable_user_defined_function/test.py::test_executable_function_non_direct_bash 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_ranged[range_hashed-True-False] 0.00s setup test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-s3_no_native_copy] 0.00s setup test_accept_invalid_certificate/test.py::test_strict_connection_reject 0.00s setup test_backup_restore_azure_blob_storage/test.py::test_backup_restore_correct_block_ids 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[flat-False-False] 0.00s teardown test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool 0.00s setup test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_ranged[range_hashed] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[cache-False-False] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_hashed-True-False] 0.00s setup test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-s3_no_native_copy] 0.00s setup test_cow_policy/test.py::test_cow_policy[cow_policy_multi_volume] 0.00s setup test_executable_user_defined_function/test.py::test_executable_function_non_direct_bash 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[hashed-False-True] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[hashed-False-False] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[flat-True-True] 0.00s setup test_ddl_alter_query/test.py::test_ddl_queue_hostname_change 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[hashed-True-False] 0.00s teardown test_file_cluster/test.py::test_count 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[hashed-False-False] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_hashed-True-False] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[direct-True-True] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_direct-True-False] 0.00s teardown test_backup_restore_azure_blob_storage/test.py::test_backup_restore_on_merge_tree 0.00s setup test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-default] 0.00s setup test_accept_invalid_certificate/test.py::test_strict_reject_with_config 0.00s setup test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-s3_native_copy] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_ranged[range_hashed-True-False] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[flat-False-True] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[direct-True-False] 0.00s setup test_attach_partition_using_copy/test.py::test_both_mergetree 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[cache-True-False] 0.00s teardown test_config_xml_main/test.py::test_xml_main_conf 0.00s teardown test_executable_user_defined_function/test.py::test_executable_function_signalled_python 0.00s setup test_executable_user_defined_function/test.py::test_executable_function_argument_python 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[direct-False-True] 0.00s teardown test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-default] 0.00s setup test_accept_invalid_certificate/test.py::test_default 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[flat-False-True] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[flat-True-False] 0.00s teardown test_accept_invalid_certificate/test.py::test_strict_connection_reject 0.00s setup test_compatibility_merge_tree_settings/test.py::test_config_overrides_compatibility 0.00s teardown test_backup_restore_azure_blob_storage/test.py::test_backup_restore_correct_block_ids 0.00s setup test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[direct-True-True] 0.00s setup test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-s3_native_copy] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[cache-False-True] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[flat-True-False] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[hashed-False-True] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[direct-False-False] 0.00s teardown test_executable_user_defined_function/test.py::test_executable_function_query_cache 0.00s setup test_executable_user_defined_function/test.py::test_executable_function_sum_json_python 0.00s teardown test_distributed_directory_monitor_split_batch_on_failure/test.py::test_distributed_background_insert_split_batch_on_failure_OFF 0.00s teardown test_executable_user_defined_function/test.py::test_executable_function_input_nullable_python 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[flat-False-False] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[direct-False-True] 0.00s teardown test_executable_user_defined_function/test.py::test_executable_function_argument_python 0.00s setup test_check_table_name_length/test.py::test_check_table_name_length 0.00s teardown test_accept_invalid_certificate/test.py::test_default 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[flat-True-True] 0.00s teardown test_backup_restore_azure_blob_storage/test.py::test_backup_restore_with_named_collection_azure_conf1 0.00s teardown test_accept_invalid_certificate/test.py::test_accept 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[cache-True-False] 0.00s teardown test_default_role/test.py::test_alter_user 0.00s teardown test_accept_invalid_certificate/test.py::test_connection_accept 0.00s setup test_file_cluster/test.py::test_format_detection 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[direct-True-False] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_ranged[range_hashed-False-False] 0.00s setup test_buffer_profile/test.py::test_default_profile 0.00s setup test_executable_user_defined_function/test.py::test_executable_function_parameter_python 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[hashed-True-False] 0.00s setup test_backup_restore_azure_blob_storage/test.py::test_backup_restore_diff_container 0.00s setup test_executable_user_defined_function/test.py::test_executable_function_send_chunk_header_python 0.00s setup test_backup_restore_azure_blob_storage/test.py::test_backup_restore_with_named_collection_azure_conf1 0.00s teardown test_executable_user_defined_function/test.py::test_executable_function_slow_python 0.00s teardown test_attach_partition_using_copy/test.py::test_not_work_on_different_disk 0.00s teardown test_custom_settings/test.py::test_custom_settings 0.00s setup test_executable_user_defined_function/test.py::test_executable_function_input_nullable_python 0.00s setup test_accept_invalid_certificate/test.py::test_connection_accept 0.00s setup test_executable_user_defined_function/test.py::test_executable_function_sum_python 0.00s teardown test_executable_user_defined_function/test.py::test_executable_function_python 0.00s teardown test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-s3_native_copy] 0.00s setup test_executable_user_defined_function/test.py::test_executable_function_python 0.00s setup test_distributed_directory_monitor_split_batch_on_failure/test.py::test_distributed_background_insert_split_batch_on_failure_ON 0.00s setup test_backup_restore_azure_blob_storage/test.py::test_backup_restore_with_named_collection_azure_conf2 0.00s setup test_executable_user_defined_function/test.py::test_executable_function_slow_python 0.00s setup test_attach_partition_using_copy/test.py::test_only_destination_replicated 0.00s teardown test_executable_user_defined_function/test.py::test_executable_function_bash 0.00s teardown test_buffer_profile/test.py::test_buffer_profile 0.00s teardown test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-default] 0.00s teardown test_executable_user_defined_function/test.py::test_executable_function_send_chunk_header_python 0.00s teardown test_executable_user_defined_function/test.py::test_executable_function_sum_json_python 0.00s setup test_executable_user_defined_function/test.py::test_executable_function_signalled_python 0.00s teardown test_accept_invalid_certificate/test.py::test_strict_reject 0.00s setup test_executable_user_defined_function/test.py::test_executable_function_query_cache 0.00s teardown test_executable_user_defined_function/test.py::test_executable_function_parameter_python =========================== short test summary info ============================ FAILED test_attach_partition_using_copy/test.py::test_all_replicated - Except... FAILED test_cow_policy/test.py::test_cow_policy[cow_policy_multi_disk] - Exce... FAILED test_ddl_worker_replicas/test.py::test_ddl_worker_replicas - Assertion... FAILED test_attach_partition_using_copy/test.py::test_both_mergetree - Except... FAILED test_cow_policy/test.py::test_cow_policy[cow_policy_multi_volume] - Ex... FAILED test_attach_partition_using_copy/test.py::test_not_work_on_different_disk FAILED test_attach_partition_using_copy/test.py::test_only_destination_replicated PASSED test_accept_invalid_certificate/test.py::test_accept PASSED test_accept_invalid_certificate/test.py::test_connection_accept PASSED test_accept_invalid_certificate/test.py::test_default PASSED test_buffer_profile/test.py::test_buffer_profile PASSED test_accept_invalid_certificate/test.py::test_strict_connection_reject PASSED test_accept_invalid_certificate/test.py::test_strict_reject PASSED test_accept_invalid_certificate/test.py::test_strict_reject_with_config PASSED test_buffer_profile/test.py::test_default_profile PASSED test_default_role/test.py::test_alter_user PASSED test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-default] PASSED test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-s3_native_copy] PASSED test_default_role/test.py::test_set_default_roles PASSED test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-s3_no_native_copy] PASSED test_executable_user_defined_function/test.py::test_executable_function_always_error_python PASSED test_default_role/test.py::test_wrong_set_default_role PASSED test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-default] PASSED test_executable_user_defined_function/test.py::test_executable_function_argument_python PASSED test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-s3_native_copy] PASSED test_executable_user_defined_function/test.py::test_executable_function_bash PASSED test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-s3_no_native_copy] PASSED test_executable_user_defined_function/test.py::test_executable_function_input_nullable_python PASSED test_executable_user_defined_function/test.py::test_executable_function_non_direct_bash PASSED test_executable_user_defined_function/test.py::test_executable_function_parameter_python PASSED test_executable_user_defined_function/test.py::test_executable_function_python PASSED test_check_table_name_length/test.py::test_backward_compatibility PASSED test_executable_user_defined_function/test.py::test_executable_function_query_cache PASSED test_check_table_name_length/test.py::test_check_table_name_length PASSED test_compatibility_merge_tree_settings/test.py::test_check_projections_compatibility PASSED test_compatibility_merge_tree_settings/test.py::test_config_overrides_compatibility PASSED test_executable_user_defined_function/test.py::test_executable_function_send_chunk_header_python PASSED test_executable_user_defined_function/test.py::test_executable_function_signalled_python PASSED test_executable_user_defined_function/test.py::test_executable_function_slow_python PASSED test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool PASSED test_executable_user_defined_function/test.py::test_executable_function_sum_json_python PASSED test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_complex[complex_key_hashed] PASSED test_executable_user_defined_function/test.py::test_executable_function_sum_python PASSED test_custom_settings/test.py::test_custom_settings PASSED test_custom_settings/test.py::test_illformed_setting PASSED test_ddl_alter_query/test.py::test_alter PASSED test_distributed_ddl_password/test.py::test_alter PASSED test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_ranged[range_hashed] PASSED test_ddl_alter_query/test.py::test_ddl_queue_hostname_change PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_cache-False-False] PASSED test_distributed_ddl_password/test.py::test_truncate PASSED test_asynchronous_metric_log_table/test.py::test_event_time_microseconds_field PASSED test_file_cluster/test.py::test_count PASSED test_file_cluster/test.py::test_format_detection PASSED test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability PASSED test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_simple[hashed] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_direct-False-False] PASSED test_concurrent_queries_for_user_restriction/test.py::test_exception_message PASSED test_backward_compatibility/test_ip_types_binary_compatibility.py::test_ip_types_binary_compatibility PASSED test_backup_restore_azure_blob_storage/test.py::test_backup_restore PASSED test_backward_compatibility/test_short_strings_aggregation.py::test_backward_compatability PASSED test_config_xml_main/test.py::test_xml_main_conf PASSED test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated PASSED test_config_yaml_merge_keys/test.py::test_yaml_merge_keys_conf PASSED test_backup_restore_azure_blob_storage/test.py::test_backup_restore_correct_block_ids PASSED test_backup_restore_azure_blob_storage/test.py::test_backup_restore_diff_container PASSED test_backup_restore_azure_blob_storage/test.py::test_backup_restore_on_merge_tree PASSED test_cleanup_after_start/test.py::test_old_dirs_cleanup PASSED test_backup_restore_azure_blob_storage/test.py::test_backup_restore_with_named_collection_azure_conf1 PASSED test_backup_restore_azure_blob_storage/test.py::test_backup_restore_with_named_collection_azure_conf2 PASSED test_config_reloader_interval/test.py::test_reload_config PASSED test_cluster_discovery/test_password.py::test_connect_with_password PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_hashed-False-False] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_ranged[range_hashed-False-False] PASSED test_cluster_discovery/test_auxiliary_keeper.py::test_cluster_discovery_with_auxiliary_keeper_startup_and_stop PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[cache-False-False] PASSED test_compression_nested_columns/test.py::test_nested_compression_codec PASSED test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas PASSED test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[direct-False-False] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[flat-False-False] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[hashed-False-False] PASSED test_distributed_directory_monitor_split_batch_on_failure/test.py::test_distributed_background_insert_split_batch_on_failure_OFF PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[cache-False-True] PASSED test_distributed_directory_monitor_split_batch_on_failure/test.py::test_distributed_background_insert_split_batch_on_failure_ON PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[direct-False-True] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[flat-False-True] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[hashed-False-True] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_cache-True-False] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_direct-True-False] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_hashed-True-False] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_ranged[range_hashed-True-False] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[cache-True-False] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[direct-True-False] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[flat-True-False] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[hashed-True-False] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[cache-True-True] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[direct-True-True] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[flat-True-True] PASSED test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[hashed-True-True] ================== 7 failed, 93 passed in 1349.09s (0:22:29) =================== Traceback (most recent call last): File "/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration/./runner", line 492, in subprocess.check_call(cmd, shell=True, bufsize=0) File "/usr/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'docker run --rm --name clickhouse_integration_tests_09sh6d --privileged --dns-search='.' --memory=30709030912 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=5dc43a6382f0 -e DOCKER_BASE_TAG=5ccda723c1fc -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=d862517635bf -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e CLICKHOUSE_USE_OLD_ANALYZER=1 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 --report-log=parallel0_0.jsonl --report-log-exclude-logs-on-passed-tests test_accept_invalid_certificate/test.py::test_accept test_accept_invalid_certificate/test.py::test_connection_accept test_accept_invalid_certificate/test.py::test_default test_accept_invalid_certificate/test.py::test_strict_connection_reject test_accept_invalid_certificate/test.py::test_strict_reject test_accept_invalid_certificate/test.py::test_strict_reject_with_config test_asynchronous_metric_log_table/test.py::test_event_time_microseconds_field test_attach_partition_using_copy/test.py::test_all_replicated test_attach_partition_using_copy/test.py::test_both_mergetree test_attach_partition_using_copy/test.py::test_not_work_on_different_disk test_attach_partition_using_copy/test.py::test_only_destination_replicated test_backup_restore_azure_blob_storage/test.py::test_backup_restore test_backup_restore_azure_blob_storage/test.py::test_backup_restore_correct_block_ids test_backup_restore_azure_blob_storage/test.py::test_backup_restore_diff_container test_backup_restore_azure_blob_storage/test.py::test_backup_restore_on_merge_tree test_backup_restore_azure_blob_storage/test.py::test_backup_restore_with_named_collection_azure_conf1 test_backup_restore_azure_blob_storage/test.py::test_backup_restore_with_named_collection_azure_conf2 'test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-default]' 'test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-s3_native_copy]' 'test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[0-s3_no_native_copy]' 'test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-default]' 'test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-s3_native_copy]' 'test_backup_restore_s3/test_throttling.py::test_backup_scheduler_settings[1-s3_no_native_copy]' test_backward_compatibility/test_ip_types_binary_compatibility.py::test_ip_types_binary_compatibility test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability test_backward_compatibility/test_short_strings_aggregation.py::test_backward_compatability test_buffer_profile/test.py::test_buffer_profile test_buffer_profile/test.py::test_default_profile test_check_table_name_length/test.py::test_backward_compatibility test_check_table_name_length/test.py::test_check_table_name_length test_cleanup_after_start/test.py::test_old_dirs_cleanup test_cluster_discovery/test_auxiliary_keeper.py::test_cluster_discovery_with_auxiliary_keeper_startup_and_stop test_cluster_discovery/test_password.py::test_connect_with_password test_compatibility_merge_tree_settings/test.py::test_check_projections_compatibility test_compatibility_merge_tree_settings/test.py::test_config_overrides_compatibility test_compression_nested_columns/test.py::test_nested_compression_codec test_concurrent_queries_for_user_restriction/test.py::test_exception_message test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool test_config_reloader_interval/test.py::test_reload_config test_config_xml_main/test.py::test_xml_main_conf test_config_yaml_merge_keys/test.py::test_yaml_merge_keys_conf 'test_cow_policy/test.py::test_cow_policy[cow_policy_multi_disk]' 'test_cow_policy/test.py::test_cow_policy[cow_policy_multi_volume]' test_custom_settings/test.py::test_custom_settings test_custom_settings/test.py::test_illformed_setting test_ddl_alter_query/test.py::test_alter test_ddl_alter_query/test.py::test_ddl_queue_hostname_change test_ddl_worker_replicas/test.py::test_ddl_worker_replicas test_default_role/test.py::test_alter_user test_default_role/test.py::test_set_default_roles test_default_role/test.py::test_wrong_set_default_role 'test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_complex[complex_key_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_ranged[range_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_simple[hashed]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_cache-False-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_cache-True-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_direct-False-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_direct-True-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_hashed-False-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_complex[complex_key_hashed-True-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_ranged[range_hashed-False-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_ranged[range_hashed-True-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[cache-False-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[cache-True-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[direct-False-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[direct-True-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[flat-False-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[flat-True-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[hashed-False-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple[hashed-True-False]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[cache-False-True]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[cache-True-True]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[direct-False-True]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[direct-True-True]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[flat-False-True]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[flat-True-True]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[hashed-False-True]' 'test_dictionaries_all_layouts_separate_sources/test_mongo.py::test_simple_ssl[hashed-True-True]' test_distributed_ddl_password/test.py::test_alter test_distributed_ddl_password/test.py::test_truncate test_distributed_directory_monitor_split_batch_on_failure/test.py::test_distributed_background_insert_split_batch_on_failure_OFF test_distributed_directory_monitor_split_batch_on_failure/test.py::test_distributed_background_insert_split_batch_on_failure_ON test_executable_user_defined_function/test.py::test_executable_function_always_error_python test_executable_user_defined_function/test.py::test_executable_function_argument_python test_executable_user_defined_function/test.py::test_executable_function_bash test_executable_user_defined_function/test.py::test_executable_function_input_nullable_python test_executable_user_defined_function/test.py::test_executable_function_non_direct_bash test_executable_user_defined_function/test.py::test_executable_function_parameter_python test_executable_user_defined_function/test.py::test_executable_function_python test_executable_user_defined_function/test.py::test_executable_function_query_cache test_executable_user_defined_function/test.py::test_executable_function_send_chunk_header_python test_executable_user_defined_function/test.py::test_executable_function_signalled_python test_executable_user_defined_function/test.py::test_executable_function_slow_python test_executable_user_defined_function/test.py::test_executable_function_sum_json_python test_executable_user_defined_function/test.py::test_executable_function_sum_python test_file_cluster/test.py::test_count test_file_cluster/test.py::test_format_detection -vvv " altinityinfra/integration-tests-runner:226bfaf75ac1 ' returned non-zero exit status 1.